Unimus forum

Hello,

I am really confused with the backup issue I am having. When the automatic 4hr backups run on all devices I get the below report and it lists all the failed devices etc. Yet when I login to Unimus and run the backup manually on any of those failed devices it works just fine?? Any help on this would be great while I scratch my head. Thanks all

Backup job started on: 163
Succeeded on: 115
Failed on: 48
Failed on devices: 48

Is it the same 48 devices each time or does it change from one run to the next?

My guess could be that your DB/Server is getting overloaded doing all the backups at the same time when doing them individually it has the resources to throw at the problem.

What is the hardware you are running Unimus on and what's the underlying DB you are using?

Typically it is always 48 failed, but I checked one from yesterday and it actually was only 21 this time. Below is hardware details

Hosted on Nutanix AHV (hardware is NX-8170-G9 - pretty sure just rebranded supermicro), and MS SQL for the DB

What is the actually hardware that Unimus is running on? CPU Cores and frequency + RAM size + disk? That model of computer has a lot of versions available. I would presume that you are running Unimus on a Virtual Machine (at least most people are), if not it would be good to know.

When you look at the failed jobs on the dashboard screen, what are the errors being shown? (you can select a job and above the list is a "Show Result" button) This will also help you diagnose what's going on. Or you can go to the Devices menu and hover your mouse over the red dots for the last failed jobs.

I talked to one of the lead devs yesterday and he wanted me to mention that most often the issue with scheduled backup which runs OK individually is radius or tacacs+ login limit or some FW connection limit.

CREDENTIALS_REFUSED is the error and we do use a TACACS account. When running them manually with the same creds it works fine, so I know its not a account issue. Below are specs. Thanks for the help

2 CPU 8 GB RAM
2 disks, 125 GB and 50 GB

The server is virtual and we added two more procs, but still same results

Are you using Discovery for the credentials?

I wonder if your TACACS server is getting overloaded with the queries...

What do you get when you enable Debug Logging and Device Output Logging?
https://wiki.unimus.net/display/UNPUB/L ... +Subsystem

This will give you a step by step view of what Unimus is doing when it's getting the "CREDENTIALS_REFUSED" error.

enabled debug and checked a couple in the logs that were failing. It looks like its only running a discovery job which is successful, but I am not seeing a backup job for them. Discovery basically just does a sh version then disconnects.

HEADER:
Address: 161.240.x.x
Zone number: 0
Job type: DISCOVERY
Job started: 2025-08-26 16:00:22.211
Job finished: 2025-08-26 16:00:29.330
DEVICE OUTPUT START:
<<SSH connect - socket>>SSH-2.0-Cisco-1.25
<<SSH disconnect - socket>><<SSH connect - session>><<SSH connect - shell channel>>
TIMESTAMP: 2025-08-26 16:00:23.554


swsdc-r1-9500#
TIMESTAMP: 2025-08-26 16:00:27.681

swsdc-r1-9500#show version
Cisco IOS XE Software, Version 17.09.05

omitted


swsdc-r1-9500#configure terminal
Enter configuration commands, one per line. End with CNTL/Z.
swsdc-r1-9500(config)#<<SSH disconnect - channel & session>>
DEVICE OUTPUT END:

Unimus will effectively login to a device twice every time you interact with a device.

The first login is the Discovery Job, basically this verifies to Unimus that the device is what it thinks it is and that it can interact intelligently with the device. We see here a successful Discovery job completion in your logs. Sometimes it's pretty easy to tell what a device is, sometimes its harder and Unimus needs to perform multiple commands.

With a successful Discovery Job, Unimus will then proceed to doing whatever it is that you are trying to do. Either running a Backup or performing a Mass Config Push job.

Could you show what comes up with a job failure? If you want, you can create a ticket in our support portal and submit the logs there if cleaning them up for public consumption is too much of a hassle.

Ive grabbed the device output logs from that most recent job and will open up a ticket. Thanks