Page 1 of 1

Time Warp: Crash 1 minute after boot from bwmon_gargoyle

Posted: Mon Mar 27, 2017 10:31 pm
by smiller
Hello folks.

Yesterday I started getting yelled at by my loving family because their internet was going up and down. (mostly down) The webui showed no uptime more than one minute before it went away and I had to log in again.

Using ssh, I got into the box and looked around. Logs up to about the 43 second mark, then it worked for a while, then a silent reboot.

I ran top with a one-second delay for a few reboot cycles. In every case, when it died the process on top was bwmon_gargoyle. After some googling I tried /etc/init.d/bwmon_gargoyle stop (and disable).

Stability returned.

Archer C7 v2 running 1.9.1. QoS enabled for download and upload - although upload may have enabled itself as I don't remember doing it.

The router gets a daily 4am reboot. Nothing had changed in my config in weeks. The only thing I can think of is that the qos/bandwidth logs had grown to some critical level that instantly killed the box when bwmon_gargoyle was started.

I took the opportunity to update to 1.9.2 with a fresh config. However, I did a backup of my config, so I have a copy of everything in /usr/data/bwmon in the crash scenario.

Has anyone seen this before? Any data I can share to help track this down?

And thanks for everything you do!

Re: Crash 1 minute after boot from bwmon_gargoyle

Posted: Tue Mar 28, 2017 3:49 am
by Lantis
You fell victim to the "time warp bug" (that's what i'm going to call it from now on so people can search for it).

BW Usage data got out of sync with real time, and when the module attempted to reload it, it caused a kernel panic.

The cause is unknown and hard to track down.

Purging /usr/data/bwmon causes the reboot loop to stop.

Solution:
Lantis wrote:Disconnect the WAN. Allow it to boot successfully. Purge /usr/data/bwmon/*. Reconnect WAN and reboot.

If disconnecting the WAN doesn't stop the boot loop, drop it into failsafe and do the same thing.

Re: Time Warp: Crash 1 minute after boot from bwmon_gargoyle

Posted: Wed Mar 29, 2017 12:40 am
by smiller
Lantis,

Thanks.

I updated the thread title to put "Time Warp" in it.

Now I'm humming to myself "Let's do the time warp again!"

Re: Time Warp: Crash 1 minute after boot from bwmon_gargoyle

Posted: Wed Mar 29, 2017 5:01 am
by Lantis

Re: Time Warp: Crash 1 minute after boot from bwmon_gargoyle

Posted: Sun Mar 10, 2019 11:12 am
by michaelmantis
Thank goodness for this forum!

Woke up today (day of daylight savings) to my router rebooting every minute.

Was able to get into the web interface (basically copied the "bandwidth.sh" link into my browser and hit refreshed non stop until I was able to get it right when the router rebooted) and hit "Delete Data" under the "Bandwidth Usage Table".

Been 15 minutes an no reboots yet. Did I pretty much solve the issue of needing to clean the data from /usr/data/bwmon (without having to SSH in?)

Re: Time Warp: Crash 1 minute after boot from bwmon_gargoyle

Posted: Sun Mar 10, 2019 1:39 pm
by RomanHK
michaelmantis wrote:Did I pretty much solve the issue of needing to clean the data from /usr/data/bwmon (without having to SSH in?)
If you do this, click Status - Bandwidth Usage - Delete Data in the GUI to solve the problem.

Re: Time Warp: Crash 1 minute after boot from bwmon_gargoyle

Posted: Sun Mar 10, 2019 9:10 pm
by michaelmantis
Quickly hitting "Status - Bandwidth Usage - Delete Data in the GUI" (within a minute of boot up) solved the problem.

Been up and running for 10 hours with no issue.