[Fixed in 1.7.1] Mass Config Push Fail

Unimus support forum
Post Reply
User avatar
lweidig
Posts: 31
Joined: Fri Jan 12, 2018 4:43 pm

Fri May 11, 2018 1:02 pm

We upgraded to 1.7.0 and rebuilt our mass config push scripts which were completely lost in the upgrade. We have a simple script that we are pushing out to some devices (20 total) and after running the job experienced the following results. From the web based UI we see:
unimusconfig.png
unimusconfig.png (7.62 KiB) Viewed 7313 times
Here are the logs:

Code: Select all

2018-05-11 07:48:29.104  INFO 586034 --- [https-jsse-nio-209.103.224.97-443-exec-7] net.unimus.business.core.CoreClientImpl  : Running device interaction for 20 devices
2018-05-11 07:48:29.187  INFO 586034 --- [https-jsse-nio-209.103.224.97-443-exec-7] net.unimus.core.api.CoreImpl             : Running configuration push to 28 devices
2018-05-11 07:48:34.646  WARN 586034 --- [interact-24] n.u.c.service.cli.CliInteractionService  : Config push to 'xxxxx1.excel.net' failed - 'Command not supported by device'!
2018-05-11 07:48:34.664  WARN 586034 --- [interact-27] n.u.c.service.cli.CliInteractionService  : Config push to 'xxxxx2.excel.net' failed - 'Command not supported by device'!
2018-05-11 07:48:34.680  WARN 586034 --- [interact-8] n.u.c.service.cli.CliInteractionService  : Config push to 'xxxxx3.excel.net' failed - 'Command not supported by device'!
2018-05-11 07:48:34.693  WARN 586034 --- [interact-26] n.u.c.service.cli.CliInteractionService  : Config push to 'xxxxx4.excel.net' failed - 'Command not supported by device'!
2018-05-11 07:48:34.720  WARN 586034 --- [interact-22] n.u.c.service.cli.CliInteractionService  : Config push to 'xxxxx5.excel.net' failed - 'Command not supported by device'!
2018-05-11 07:48:34.738  WARN 586034 --- [interact-21] n.u.c.service.cli.CliInteractionService  : Config push to 'xxxxx6.excel.net' failed - 'Command not supported by device'!
2018-05-11 07:48:34.756  WARN 586034 --- [interact-16] n.u.c.service.cli.CliInteractionService  : Config push to 'xxxxx7.excel.net' failed - 'Command not supported by device'!
2018-05-11 07:48:34.834  WARN 586034 --- [interact-25] n.u.c.service.cli.CliInteractionService  : Config push to 'xxxxx8.excel.net' failed - 'Command not supported by device'!
2018-05-11 07:48:34.837  WARN 586034 --- [interact-28] n.u.c.service.cli.CliInteractionService  : Config push to 'xxxxx9.excel.net' failed - 'Command not supported by device'!
2018-05-11 07:48:34.981  WARN 586034 --- [interact-23] n.u.c.service.cli.CliInteractionService  : Config push to 'xxxxx10.excel.net' failed - 'Command not supported by device'!
2018-05-11 07:48:34.996  WARN 586034 --- [interact-11] n.u.c.service.cli.CliInteractionService  : Config push to 'xxxxx11.excel.net' failed - 'Command not supported by device'!
The 11 failing devices are fine, they already had the code and the command fails with a duplicate attempt to add. The problems are:
  • What happened to the other 8 devices - UI only accounts for 12 of 20
  • The log shows push to 28 devices! How do we know the other 8 it just decided to pick
  • The last run never updated, but that may be because the job is still hanging out there or crashed (though nothing logged)
Thanks!
User avatar
Tomas
Posts: 1206
Joined: Sat Jun 25, 2016 12:33 pm

Fri May 11, 2018 1:17 pm

Could you please run Unimus in debug mode and try to replicate an incorrect run like this?

Code: Select all

service unimus stop
cd /opt/unimus
java -jar Unimus.jar --logging.level.net.unimus.core=DEBUG
Please send me the logs afterwards.

Thanks!
User avatar
lweidig
Posts: 31
Joined: Fri Jan 12, 2018 4:43 pm

Fri May 11, 2018 3:10 pm

Sent log in private message.
User avatar
Tomas
Posts: 1206
Joined: Sat Jun 25, 2016 12:33 pm

Fri May 11, 2018 3:23 pm

Got the logs, thank you.

It seems Unimus internal components are not communicating properly to each other.
We are investigating.
The log shows push to 28 devices! How do we know the other 8 it just decided to pick?
The Config Push handler receives some devices twice.
(internally, 8 devices are duplicated for some reason)
The last run never updated, but that may be because the job is still hanging out there or crashed (though nothing logged)
The job is timing out, this is why the finish timestamp is not updating.
It will actually time-out and the timestamp will update, it will just take A LONG time.
User avatar
Tomas
Posts: 1206
Joined: Sat Jun 25, 2016 12:33 pm

Fri May 11, 2018 5:11 pm

We found the issue, it was caused by devices that have multiple available connectors (SSH + Telnet).
The push invoker would tell the push handler to push twice to those devices (once per connector).

We fixed this, so the push invoker will properly use the most secure available connector for each device.
(and eliminated the bugs / duplicate calls)

Fix will be released in 1.7.1 (hotfix release going out today).

Please let us know if this fixed your issue.

Thanks!
User avatar
Tomas
Posts: 1206
Joined: Sat Jun 25, 2016 12:33 pm

Fri May 11, 2018 7:28 pm

Just an update, 1.7.1 is out:
viewtopic.php?p=1137#p1137
User avatar
lweidig
Posts: 31
Joined: Fri Jan 12, 2018 4:43 pm

Fri May 11, 2018 8:08 pm

These issues were indeed resolved by 1.7.1. Thanks!
Post Reply