Page 1 of 1

Cisco ASA multiple context issues

Posted: Thu Feb 16, 2023 8:46 pm
by dahook
Hi!

We purchased Unimus about a month ago and we really like it. However, we've discovered that backups of Cisco ASA's sometimes fail.
The backup also takes a _really_ long time.
We have several large multi-context ASA's and I believe the problem might be that you seem to paginate the output. I did a trace, which shows that you are doing a "more system:running-config". It is not preceeded by a "terminal pager 0" which means you have to send something in order for it to paginate. That complicates things, and it is also error prone. The trace log also shows the <--- More ---> lines which confirms that you are indeed paging.

Can you please add "terminal pager 0" as a command before the "more system:running-config"? This will eliminate the need for paging the output.

Reference output below:

Code: Select all

fw1/pri/act/admin# changeto system

fw1/pri/act# more system:running-config

TIMESTAMP: 2023-02-16 21:15:41.156
: Saved
: 
: Serial Numbe....
....
....
fw1/pri/act# <<SSH disconnect - channel & session>>
DEVICE OUTPUT END:
Sample "terminal pager 0". It does not return anything, just shows the prompt again.

Code: Select all

fw1/pri/act/admin# terminal pager 0
fw1/pri/act/admin# 


One more thing regarding this. Since we have many contexts it is not really practical so have everything bundled under the same device. The diff gets gigantic, and we also need to limit access to configurations for different users which we can't do right now.

I tried adding a single context as a device, but that fails. The reason is that you are checking if the ASA is multicontext, and if it is then you are trying to switch to the system context. Since it it not possible to switch to the system context when connected via SSH directly into a non admin-context that does not work. Also, it would of course cause the backup to contain all contexts under a single device anyway.

It would be really nice if you could check that you are in the "admin" context, and if you are then you can switch to the system context. In other cases you should just backup the context you are in, similar to a backup of a non-multicontext ASA.

I understand that this would mean a little more work, but it would be very helpful to us.

Sample output when inside "admin" context". The asterisk (*) is indicative of it being the admin context, from which it is possible to switch to the system context.

Code: Select all

/pri/act/admin# show context 
Context Name      Class                Interfaces           Mode         URL
*admin            default              Management0/0        Routed       disk0:/admin.cfg
fw1/pri/act/admin# 
Sample output when in a "customer" context.

Code: Select all

fw1/cust1# show context 
Context Name      Class                Interfaces           Mode         URL
 cust1              default              Port-channel9.1652,   Routed       disk0:/cust1.cfg
                                       1870-1877           
fw1/cust1#
 

Re: Cisco ASA multiple context issues

Posted: Thu Feb 16, 2023 9:50 pm
by Tomas
Hi, let me address the individual issues separately.
dahook wrote:
Thu Feb 16, 2023 8:46 pm
we've discovered that backups of Cisco ASA's sometimes fail
What's the job failure reasons for these backup jobs?
dahook wrote:
Thu Feb 16, 2023 8:46 pm

The backup also takes a _really_ long time.
We have several large multi-context ASA's and I believe the problem might be that you seem to paginate the output. I did a trace, which shows that you are doing a "more system:running-config". It is not preceeded by a "terminal pager 0" which means you have to send something in order for it to paginate. That complicates things, and it is also error prone. The trace log also shows the <--- More ---> lines which confirms that you are indeed paging.

Can you please add "terminal pager 0" as a command before the "more system:running-config"? This will eliminate the need for paging the output.
Paging is actually quite desired for Unimus, as it gives Unimus a constant "sync" point in device communication. Unimus knows that a device is outputting and wants to output more. This ties into a long discussion on job timeouts, how Unimus knows if a device gets stuck during output (we do see this a lot actually, and have provisions in the code to detect such situation), how Unimus understands that an output is complete, etc.

Back to the topic at hand - which Unimus version are you running? There was a bug in 2.3.0 Betas (not the GA release) which indeed caused excessive waits during various device communication points. Could you please upgrade to the latest (2.3.1) GA release and let us know if this still occurs?

If the backup still takes very long on 2.3.1, could you please submit a Support Ticket with logs (both the debug log and the device output log) attached?
dahook wrote:
Thu Feb 16, 2023 8:46 pm
One more thing regarding this. Since we have many contexts it is not really practical so have everything bundled under the same device. The diff gets gigantic, and we also need to limit access to configurations for different users which we can't do right now.

I tried adding a single context as a device, but that fails. The reason is that you are checking if the ASA is multicontext, and if it is then you are trying to switch to the system context. Since it it not possible to switch to the system context when connected via SSH directly into a non admin-context that does not work. Also, it would of course cause the backup to contain all contexts under a single device anyway.

It would be really nice if you could check that you are in the "admin" context, and if you are then you can switch to the system context. In other cases you should just backup the context you are in, similar to a backup of a non-multicontext ASA.

I understand that this would mean a little more work, but it would be very helpful to us.
This is a very nice suggestion, and I agree there should be options to back up just a single context or all of them. I will discuss with the dev team what the exact internal behavior of our current multi-context ASA driver is, and if we can easily adjust the driver to behave like you described.

Re: Cisco ASA multiple context issues

Posted: Fri Feb 17, 2023 4:44 pm
by dahook
Hi,

Thank you for a very complete answer. I never though about paging being useful when extracting stuff from interactive shells but I get your point :-). I will have to investigate regarding failures a little more, I might have been a bit misinformed there. Not sure there were failures, it could be related to the recent upgrade. We are running 2.3.1, upgraded from 2.2.4.

As an example, an ASA with about 40k config lines takes ~2 min 40 seconds. "Long time" is relative, with paging in mind maybe that is to be expected. Not really an issue to be honest, just thought that it would be faster since a dump with paging disabled takes a few seconds.

Completely off topic, I noticed that we get an SQL error when clicking "Show Usage" for credentials after the upgrade. We are still using HSQL, I will reinstall and use an external db soon and I don't see it as a problem. Is it something you've seen elsewhere?

Re: Cisco ASA multiple context issues

Posted: Fri Feb 17, 2023 4:57 pm
by Vik@Unimus
Hello,

Would you mind submitting a support ticket via our Portal regarding the aforementioned SQL error and include Unimus log with it? You can locate Unimus log at

Linux - /var/log/unimus
Windows - C:\ProgramData\Unimus\log

I will forward information to our team to take a closer look at that.

Re: Cisco ASA multiple context issues

Posted: Sun Feb 19, 2023 2:14 pm
by dahook
Certainly! Ticket created :-)