Confd crashes when configuring snmp engine-id

tmmachado · February 27, 2025, 8:22pm

We did a Confd upgrade from version 7.2.6 to 8.0.6. After this, an error started to occur when the snmp engine-id and aaa authentication-order are configured on same commit.

The yang related to engine-id:

augment /snmp:snmp {
  container agent {
    container engine-id {
        reference "SNMP-FRAMEWORK-MIB.snmpEngineID";
        tailf:info
          "Local SNMP engine's administratively-unique identifier";
        leaf enterprise-number {
          type uint32;
          default 3709;
        }
        choice method {
          default from-mac-address;
          leaf from-ip {
            tailf:info
              "Set IP address for the engine ID";
            type inet:ip-address;
          }
          leaf from-mac-address {
            tailf:info
              "Set MAC address for the engine ID";
            type yang:mac-address;
            tailf:default-ref '/defaultMacAddress';
          }
          leaf from-text {
            tailf:info
              "Set Text address for the engine ID";
            type string {
              length 1..27;
            }
          }
          leaf other {
            type string {
              pattern "[0-9a-fA-F]{2}(:[0-9a-fA-F]{2}){0,27}";
            }
          }
        }    
     }                        
     
     
 leaf defaultMacAddress {
   tailf:hidden factory-config;
   type yang:mac-address;
 }

We have a hidden leaf defaultMacAddress, that is set on initialization using a static xml. The engine-id is a choice with default “from-mac-address”, and it uses a default-ref to this leaf. It works.

We have another configuration related to “aaa authentication-order”. If we configure engine-id and aaa authentication-order on same commit, a crash log is generated. It happens always (the crash log). And sometimes (not always, but frequently), the Confd crashes during the commit and CLI presents the message “Internal error: Restarting CLI…”. It’s needed to restart the equipment to recover. In this situation, “confd --status” doesn’t answer and it’s needed to interrupt with ctrl-c (only the first lines are shown).

The aaa config is: aaa authentication-order [ local ]
But this is the config already commited. If i try to commit only this aaa line, the confd reports “no modifications to commit” (it’s ok), but when this line is inserted together with the engine-id config, tha crash happens.

The aaa yang is:


     leaf-list authentication-order {                                                                                                                                                                           
            ordered-by user;
            tailf:cli-full-command;
            tailf:info "Authentication order configuration";
            type enumeration {
                enum local {
                    tailf:code-name "local-order";
                }
                enum radius {
                    tailf:code-name "radius-order";
                }   
                enum tacacs{
                    tailf:code-name "tacacs-order";
                }   
            }   
        }

I check on some debugs that our code related to aaa is not called.

Below, the crash log:


** Reason for termination ==
** {function_clause,
       [{cdb_agent,with_agent_call,
            [notab,{rreq,default_db,lookup,[[[516812954|132886733]],false]}],
            [{file,"cdb_agent.erl"},{line,145}]},
        {cdb,get_elem,7,[{file,"cdb.erl"},{line,467}]},
        {cs_dbm,do_op,8,[{file,"cs_dbm.erl"},{line,510}]},
        {cs_dbm,rs_db_op,6,[{file,"cs_dbm.erl"},{line,194}]},
        {cst_get_delem,do_sh_get_delem,3,
            [{file,"cst_get_delem.erl"},{line,312}]},
        {cst_get_delem,sh_get_delem3,3,[{file,"cst_get_delem.erl"},{line,62}]},
        {cst_lib,sh_get_default_ref_value,2,[{file,"cst_lib.erl"},{line,874}]},
        {cs_trans_diff_iterate,get_default_set_value,4,
            [{file,"cs_trans_diff_iterate.erl"},{line,1455}]},
        {cs_trans_diff_iterate,get_op_val,9,
            [{file,"cs_trans_diff_iterate.erl"},{line,907}]},
        {cs_trans_diff_iterate,sh_diff_iterate_x,11,
            [{file,"cs_trans_diff_iterate.erl"},{line,232}]},
        {xds_ram,tts_fold_children,7,[{file,"xds_ram.erl"},{line,1466}]},
        {xds_ram,fold_children,4,[{file,"xds_ram.erl"},{line,490}]},
        {cs_trans_diff_iterate,sh_diff_iterate_x,11,
            [{file,"cs_trans_diff_iterate.erl"},{line,523}]},
        {xds_ram,tts_fold_children,7,[{file,"xds_ram.erl"},{line,1466}]},
        {xds_ram,fold_children,4,[{file,"xds_ram.erl"},{line,490}]},
        {cs_trans_diff_iterate,sh_diff_iterate_x,11,
            [{file,"cs_trans_diff_iterate.erl"},{line,523}]},
        {xds_ram,tts_fold_children,7,[{file,"xds_ram.erl"},{line,1466}]},
        {xds_ram,fold_children,4,[{file,"xds_ram.erl"},{line,490}]},
        {cs_trans_diff_iterate,sh_diff_iterate_x,11,
            [{file,"cs_trans_diff_iterate.erl"},{line,523}]},
        {xds_ram,tts_fold_children,7,[{file,"xds_ram.erl"},{line,1466}]},
        {xds_ram,fold_children,4,[{file,"xds_ram.erl"},{line,490}]},
        {cs_trans_diff_iterate,sh_diff_iterate_x,11,
            [{file,"cs_trans_diff_iterate.erl"},{line,523}]},
        {cs_trans_diff_iterate,sh_diff_iterate,7,
            [{file,"cs_trans_diff_iterate.erl"},{line,154}]},
        {aaa_server,interesting_event,1,
            [{file,"aaa_server.erl"},{line,4335}]}]}

=CRASH REPORT==== 27-Feb-2025::16:50:59.046482 ===
  crasher:
    initial call: aaa_server:init/1
    pid: <0.22566.0>
    registered_name: aaa_server
    exception error: no function clause matching 
                     cdb_agent:with_agent_call(notab,
                                               {rreq,default_db,lookup,
                                                [[[516812954|132886733]],
                                                 false]}) (cdb_agent.erl, line 145)
      in function  cdb:get_elem/7 (cdb.erl, line 467)
      in call from cs_dbm:do_op/8 (cs_dbm.erl, line 510)
      in call from cs_dbm:rs_db_op/6 (cs_dbm.erl, line 194)
      in call from cst_get_delem:do_sh_get_delem/3 (cst_get_delem.erl, line 312)
      in call from cst_get_delem:sh_get_delem3/3 (cst_get_delem.erl, line 62)
      in call from cst_lib:sh_get_default_ref_value/2 (cst_lib.erl, line 874)
      in call from cs_trans_diff_iterate:get_default_set_value/4 (cs_trans_diff_iterate.erl, line 1455)
      in call from cs_trans_diff_iterate:get_op_val/9 (cs_trans_diff_iterate.erl, line 907)
      in call from cs_trans_diff_iterate:sh_diff_iterate_x/11 (cs_trans_diff_iterate.erl, line 232)
      in call from xds_ram:tts_fold_children/7 (xds_ram.erl, line 1466)
      in call from xds_ram:fold_children/4 (xds_ram.erl, line 490)
      in call from cs_trans_diff_iterate:sh_diff_iterate_x/11 (cs_trans_diff_iterate.erl, line 523)
      in call from xds_ram:tts_fold_children/7 (xds_ram.erl, line 1466)
      in call from xds_ram:fold_children/4 (xds_ram.erl, line 490)
      in call from cs_trans_diff_iterate:sh_diff_iterate_x/11 (cs_trans_diff_iterate.erl, line 523)
      in call from xds_ram:tts_fold_children/7 (xds_ram.erl, line 1466)
      in call from xds_ram:fold_children/4 (xds_ram.erl, line 490)
      in call from cs_trans_diff_iterate:sh_diff_iterate_x/11 (cs_trans_diff_iterate.erl, line 523)
      in call from xds_ram:tts_fold_children/7 (xds_ram.erl, line 1466)
      in call from xds_ram:fold_children/4 (xds_ram.erl, line 490)
      in call from cs_trans_diff_iterate:sh_diff_iterate_x/11 (cs_trans_diff_iterate.erl, line 523)
      in call from cs_trans_diff_iterate:sh_diff_iterate/7 (cs_trans_diff_iterate.erl, line 154)
      in call from aaa_server:interesting_event/1 (aaa_server.erl, line 4335)
    ancestors: [confd_second_sup,confd_sup,<0.48.0>]
    message_queue_len: 1
    messages: [confd_cfg_updated]
    links: [<0.93.0>]
    dictionary: [{'$confd_cfg_validateUtf8',true}]
    trap_exit: true
    status: running
    heap_size: 75113
    stack_size: 27
    reductions: 287540
  neighbours:

=SUPERVISOR REPORT==== 27-Feb-2025::16:50:59.056236 ===
    supervisor: {local,confd_second_sup}
    errorContext: child_terminated
    reason: {function_clause,
                [{cdb_agent,with_agent_call,
                     [notab,
                      {rreq,default_db,lookup,
                          [[[516812954|132886733]],false]}],
                     [{file,"cdb_agent.erl"},{line,145}]},
                 {cdb,get_elem,7,[{file,"cdb.erl"},{line,467}]},
                 {cs_dbm,do_op,8,[{file,"cs_dbm.erl"},{line,510}]},
                 {cs_dbm,rs_db_op,6,[{file,"cs_dbm.erl"},{line,194}]},
                 {cst_get_delem,do_sh_get_delem,3,
                     [{file,"cst_get_delem.erl"},{line,312}]},
                 {cst_get_delem,sh_get_delem3,3,
                     [{file,"cst_get_delem.erl"},{line,62}]},
                 {cst_lib,sh_get_default_ref_value,2,
                     [{file,"cst_lib.erl"},{line,874}]},
                 {cs_trans_diff_iterate,get_default_set_value,4,
                     [{file,"cs_trans_diff_iterate.erl"},{line,1455}]},
                 {cs_trans_diff_iterate,get_op_val,9,
                     [{file,"cs_trans_diff_iterate.erl"},{line,907}]},
                 {cs_trans_diff_iterate,sh_diff_iterate_x,11,
                     [{file,"cs_trans_diff_iterate.erl"},{line,232}]},
                 {xds_ram,tts_fold_children,7,
                     [{file,"xds_ram.erl"},{line,1466}]},
                 {xds_ram,fold_children,4,[{file,"xds_ram.erl"},{line,490}]},
                 {cs_trans_diff_iterate,sh_diff_iterate_x,11,
                     [{file,"cs_trans_diff_iterate.erl"},{line,523}]},
                 {xds_ram,tts_fold_children,7,
                     [{file,"xds_ram.erl"},{line,1466}]},
                 {xds_ram,fold_children,4,[{file,"xds_ram.erl"},{line,490}]},
                 {cs_trans_diff_iterate,sh_diff_iterate_x,11,
                     [{file,"cs_trans_diff_iterate.erl"},{line,523}]},
                 {xds_ram,tts_fold_children,7,
                     [{file,"xds_ram.erl"},{line,1466}]},
                 {xds_ram,fold_children,4,[{file,"xds_ram.erl"},{line,490}]},
                 {cs_trans_diff_iterate,sh_diff_iterate_x,11,
                     [{file,"cs_trans_diff_iterate.erl"},{line,523}]},
                 {xds_ram,tts_fold_children,7,
                     [{file,"xds_ram.erl"},{line,1466}]},
                 {xds_ram,fold_children,4,[{file,"xds_ram.erl"},{line,490}]},
                 {cs_trans_diff_iterate,sh_diff_iterate_x,11,
                     [{file,"cs_trans_diff_iterate.erl"},{line,523}]},
                 {cs_trans_diff_iterate,sh_diff_iterate,7,
                     [{file,"cs_trans_diff_iterate.erl"},{line,154}]},
                 {aaa_server,interesting_event,1,
                     [{file,"aaa_server.erl"},{line,4335}]}]}
    offender: [{pid,<0.22566.0>},
               {id,aaa_server},
               {mfargs,{aaa_server,start_link,[]}},
               {restart_type,permanent},
               {shutdown,2000},
               {child_type,worker}]

Do you have any idea about this problem?

I remove the default-ref from engine-id yang and the problem doesn’t happen. Use choice with default-ref is not supported anymore?

What’s the relation between the engine-id config and aaa?

When engine-id is configured, this path is set:
/confd_dyncfg:confdConfig/snmpAgent/snmpEngine/snmpEngineID

Thanks.
Best regards,
Tatiane

fnchooft · February 28, 2025, 8:42pm

Afternoon Tatiane, we will take a look at this issue. Thank you for providing the crash-log which is very helpful.

Kind regards,
Fabian

fnchooft · March 6, 2025, 11:48am

Hi Tatiane, I picked up your issue Confd crashes when configuring snmp engine-id, however I would need some additional information in order to debug further.

Could you provide some more information about the exact models you are augmenting?
Additionally, in your Makefiles there are some additional rules to compile the models and these would be very helpful in order to find out what exactly is going on.

I started my analysis based on the 6-dyncfg example from the examples-directory. The easiest way for us to get to a better understanding would be to adapt this example to fail in the same manner which you are describing.

Kind regards,

Fabian

fnchooft · March 13, 2025, 1:47pm

Morning Tatiane, were you able to reproduce the issue using the snmp-example?

Kind regards,

Fabian

tmmachado · March 13, 2025, 8:03pm

Hi Fabian, I didn’t reproduce the problem with the example yet. I’ll try to do it as soon as possible.

Best regards,

Tatiane.

Topic		Replies	Views
How to set up snmp v3 properly Other Northbound Interfaces	4	4334	July 10, 2017
Typedef/leafref and SNMP Other Northbound Interfaces	5	2070	July 7, 2016
Leaf-list SNMP response Other Northbound Interfaces	1	531	October 28, 2022
Confd is crashing with "Internal error: Supervision terminated" Other Northbound Interfaces	4	1880	April 5, 2021
Internal Error , when load merge cfg in confd-state version 7.5.2.1 YANG	3	640	January 18, 2022

Confd crashes when configuring snmp engine-id

Related topics