Page 1 of 1
[Fixed in 2.1.0] Config diff is slow in 2.0.x
Posted: Mon Jun 01, 2020 2:15 pm
by petr
Hello,
when I try to do a config diff on FortiMail device in Unimus 2.0.x it takes around 30 seconds, but in unimus 1.10.4 it is almost instantaneous. The config has around 37000 lines(1092kB). If I try diff on FortiGate config, which has around 10000 lines(302kB), It takes less than 1 second. So I guess the problem is with bigger config files? It also makes the backup process much slower. Is there anything I can change in configuration to make it faster or is it a problem in version 2.0?
Re: Config diff is slow in 2.0.x
Posted: Tue Jun 02, 2020 1:33 pm
by Tomas
Hi,
Confirming this as a performance problem with large backup diffs in 2.0.0 / 2.0.1.
We are looking into this and will fix it in either 2.0.2 or 2.0.3 at latest.
Re: Config diff is slow in 2.0.x
Posted: Mon Jul 27, 2020 8:48 am
by petr
Hi,
I have seen this in 2.0.3-Beta2 changelog:
- Improved performance of diff generation - specially visible on large diffs
So I thought I would try it and see if it is faster. But even in this version the diff still takes about 30s to make. So it looks like nothing changed.
Re: Config diff is slow in 2.0.x
Posted: Mon Aug 17, 2020 6:53 am
by petr
Hi,
after trying 2.0.4, I have to say, that unfortunately nothing changed and the slow config diff is still happening...
Re: [Fixed in 2.1.0] Config diff is slow in 2.0.x
Posted: Wed Sep 08, 2021 7:09 pm
by Tomas
We have found one additional case where our diffing logic could be slow - on very large "one-sided" diffs. For example, if many lines (100s) changed in the newer revision as compared to an old revision.
We have optimized this part of the diffing code, and this should now behave much better.
Re: [Fixed in 2.1.0] Config diff is slow in 2.0.x
Posted: Mon Sep 13, 2021 8:29 am
by petr
Well after trying the diff in 2.1.2 unfortunately nothing has changed. The fortimail diff still takes about a minute while using 100% CPU. What I found out is, that it depends on how many certificates are stored in the config file. For each certificate in the config file the diff takes much longer.
Re: [Fixed in 2.1.0] Config diff is slow in 2.0.x
Posted: Mon Sep 13, 2021 4:03 pm
by Vik@Unimus
Thank you for letting us know. I have created an internal ticket with the current situation summary, so that we can go from there. We are working on another algorithm, which might help, but I don't have an ETA I could offer right now.
I spoke with our developer who was troubleshooting the issue with you back in January and he mentioned we couldn't replicate the same CPU usage numbers as you, thus I would like to also ask what CPU you are running, so that we have a better idea what HW is causing 100% load during processing diffs.