[Solved] Terminal codes & format deltas showing in backup
It appears to be happening primarily on generation 2 & 3 ZL series switches; my gen 2 is running K16 (latest major) and gen 3 running K15 code. It seems to be random from day to day; however, the error has an affinity for switches with >1000 line configurations and those with long line (screen wrapped) output.
I'm stepping through other gear...
Another thought:
The regex --MORE-- match in this log:
The Procurve command: will prevent more "press-any-key" style interaction. This has helped me in the past when using plink/Rancid/powershell scripts to strip extraneous terminal codes (e.g. --MORE--) or anything that would normally require user interaction for continuous "show running-config" output.
I'm stepping through other gear...
Another thought:
The regex --MORE-- match in this log:
Code: Select all
2017-03-03 15:46:49.436 WARN 7604 --- [ backup-45] n.u.c.drivers.hp.HpProcurveBackupDriver : Unable to download backup - 'Expect operation fails (timeout: 20000 ms) for matcher: anyOf(regexp('\QMySwitchName\E# $'),regexp('-- MORE --'))'!
Code: Select all
no page
VT100 sequences
For VT100 control sequences, we have a global filter that filters them before data from the SSH session even make it into any code that interacts with SSH. The filter implementation has not been changed since 0.1.3 when it was implemented.
So it is very strange that you are seeing issues after migration from 0.2 to 0.3.
Are the issues consistent across backups on a single device?
So if the control sequence is present on a device in one backup - is it also in another?
Or is it totally random, and they appear arbitrarily across devices and backups?
'- more -' and pagination
As for pagination and '- more -', we use pagination on purpose.
It is very handy as a synchronization point - the code actually expects it, because it signifies that device is outputting as expected.
(since its a well know sequence, which can be depended upon to appear every now and then)
Unimus actually uses a quite large terminal size (which means pagination doesnt appear very often) - but it it still very useful .
For VT100 control sequences, we have a global filter that filters them before data from the SSH session even make it into any code that interacts with SSH. The filter implementation has not been changed since 0.1.3 when it was implemented.
So it is very strange that you are seeing issues after migration from 0.2 to 0.3.
Are the issues consistent across backups on a single device?
So if the control sequence is present on a device in one backup - is it also in another?
Or is it totally random, and they appear arbitrarily across devices and backups?
'- more -' and pagination
As for pagination and '- more -', we use pagination on purpose.
It is very handy as a synchronization point - the code actually expects it, because it signifies that device is outputting as expected.
(since its a well know sequence, which can be depended upon to appear every now and then)
Unimus actually uses a quite large terminal size (which means pagination doesnt appear very often) - but it it still very useful .
Are the issues consistent across backups on a single device?
No. The issues appear on multiple devices; however, the devices I'm seeing this happen on are HPE Procurves running two versions of code. I count the following gear:model:code-(date) with this manifestation:
E1:J9850A:KB.15.18.0006-(Mar6-7&7-8&8-9)
DCSw1:J8698A:K.16.02.0010-(Mar8-9)
So if the control sequence is present on a device in one backup - is it also in another?
Sometimes they appear in different places of the config - from day to day.
Or is it totally random, and they appear arbitrarily across devices and backups?
Sometimes in the middle of the config and the very end. It appears to be random areas toward the half-way point to the end of the config.
-----
I came across another issue while stepping through my backups (I wish there was a search function!). I have an HP j9726a that is being discovered correctly, but not being backed up. It's similar to the ASA issue described in a previous post.
No. The issues appear on multiple devices; however, the devices I'm seeing this happen on are HPE Procurves running two versions of code. I count the following gear:model:code-(date) with this manifestation:
E1:J9850A:KB.15.18.0006-(Mar6-7&7-8&8-9)
DCSw1:J8698A:K.16.02.0010-(Mar8-9)
So if the control sequence is present on a device in one backup - is it also in another?
Sometimes they appear in different places of the config - from day to day.
Or is it totally random, and they appear arbitrarily across devices and backups?
Sometimes in the middle of the config and the very end. It appears to be random areas toward the half-way point to the end of the config.
-----
I came across another issue while stepping through my backups (I wish there was a search function!). I have an HP j9726a that is being discovered correctly, but not being backed up. It's similar to the ASA issue described in a previous post.
Code: Select all
2017-03-09 14:16:15.038 INFO 7604 --- [nio-8085-exec-1] net.unimus.service.DeviceService : Attempt to run 1 backup(s)
2017-03-09 14:16:15.051 INFO 7604 --- [nio-8085-exec-1] net.unimus.business.core.Core : Performing backup on 1 device(s)
2017-03-09 14:16:37.929 WARN 7604 --- [ backup-29] n.u.c.drivers.hp.HpProcurveBackupDriver : Unable to download backup - 'Expect operation fails (timeout: 20000 ms) for matcher: anyOf(regexp('\QSw6\E# $'),regexp('-- MORE --'))'!
2017-03-09 14:17:05.824 WARN 7604 --- [ backup-5] n.u.c.drivers.hp.HpProcurveBackupDriver : Unable to download backup - 'Expect operation fails (timeout: 20000 ms) for matcher: anyOf(regexp('\QSw6\E# $'),regexp('-- MORE --'))'!
I have prepared a dev-build of 0.3.2 with full debugging enabled.
This will log A LOT of data into the log file (including the full device communication).
It can be downloaded here.
1) Can you please send me logs from the communication with the j9726a thats not being backed up correctly?
(please strip any sensitive data from the logs)
2) What search function you mentioned was missing?
We will gladly add it
3) This request is a more complicated - if you could catch one backup that includes the VT100 control characters while running the debug version and send me the logs, it would help greatly.
Feel free to PM me the logs, so they are not publicly available.
This will log A LOT of data into the log file (including the full device communication).
It can be downloaded here.
1) Can you please send me logs from the communication with the j9726a thats not being backed up correctly?
(please strip any sensitive data from the logs)
2) What search function you mentioned was missing?
We will gladly add it
3) This request is a more complicated - if you could catch one backup that includes the VT100 control characters while running the debug version and send me the logs, it would help greatly.
Feel free to PM me the logs, so they are not publicly available.
Thanks Thomas,
Unfortunately, I still cannot access your owncloud. I'm presented with this error: 1) 2017-03-14 13:50:38.122 INFO 7604 --- [nio-8085-exec-9] net.unimus.service.DeviceService : Attempt to run 1 backup(s)
2017-03-14 13:50:38.134 INFO 7604 --- [nio-8085-exec-9] net.unimus.business.core.Core : Performing backup on 1 device(s)
2017-03-14 13:51:00.969 WARN 7604 --- [ backup-29] n.u.c.drivers.hp.HpProcurveBackupDriver : Unable to download backup - 'Expect operation fails (timeout: 20000 ms) for matcher: anyOf(regexp('\QMySwitchNameHere\E# $'),regexp('-- MORE --'))'!
2) For a search function: Backups --> Search allows me to search my list of _devices_. Should I search the content of a backup selection for the word '[100;1H' (e.g. an escape code referenced this trouble), I would not have to step through every date iteration of backup-per-device. It would be a quick, google-esque, search.
3) Will do. Once I can download the debug build, I will run it in a sandbox and provide you with the logs via PM.
Unfortunately, I still cannot access your owncloud. I'm presented with this error: 1) 2017-03-14 13:50:38.122 INFO 7604 --- [nio-8085-exec-9] net.unimus.service.DeviceService : Attempt to run 1 backup(s)
2017-03-14 13:50:38.134 INFO 7604 --- [nio-8085-exec-9] net.unimus.business.core.Core : Performing backup on 1 device(s)
2017-03-14 13:51:00.969 WARN 7604 --- [ backup-29] n.u.c.drivers.hp.HpProcurveBackupDriver : Unable to download backup - 'Expect operation fails (timeout: 20000 ms) for matcher: anyOf(regexp('\QMySwitchNameHere\E# $'),regexp('-- MORE --'))'!
2) For a search function: Backups --> Search allows me to search my list of _devices_. Should I search the content of a backup selection for the word '[100;1H' (e.g. an escape code referenced this trouble), I would not have to step through every date iteration of backup-per-device. It would be a quick, google-esque, search.
3) Will do. Once I can download the debug build, I will run it in a sandbox and provide you with the logs via PM.
I created a ticket for 2:
https://tracker.unimus.net/browse/UN-156
For 1 and 3, here is the build on Google Drive:
https://drive.google.com/open?id=0B6GBO ... jlXZFd5MzQ
Please let me know if you can get it and run it, and send me the logs for 1 and 3 from the debug build if so.
Thanks in advance!
https://tracker.unimus.net/browse/UN-156
For 1 and 3, here is the build on Google Drive:
https://drive.google.com/open?id=0B6GBO ... jlXZFd5MzQ
Please let me know if you can get it and run it, and send me the logs for 1 and 3 from the debug build if so.
Thanks in advance!