|
Joined: Nov 2001
Posts: 5,906 Likes: 8
Former ChatMaster ChatMaster - 5,000
|
OP
Former ChatMaster ChatMaster - 5,000
Joined: Nov 2001
Posts: 5,906 Likes: 8 |
Well, that was a tough one. I'm sorry for the lockup that we experienced yesterday. Among other things, I was unaware of it for the first three hours. I was actually programming a new REGION ROSTER feature and wasn't checking my email.
Now -- here's the problem. In a nutshell we were bombarded by bots and spammers from Thailand and the Philippines. They overwhelmed the server and it shut down. Once I dicovered this, we rebooted and it came back up okay.
In the almost three hour analysis that followed, I finally discovered that a major file was missing from our server. It's the one that lists the IP addressess of the bad guys. And then it prevents them from accessing our server. It contains the addresses of a few thousand addresses that are automatically blocked.
I have reinstalled that file and it appears that we might be secure again. But -- I don't know for sure if that was the single root cause or not. We'll just have to wait and see.
In the meantime, another server administrator who had recommended that we move to a new host, backed off when I gave him the details of our "attack". He said that this probably would have taken their server down too.
So we march on.
Bill Barker Previous VCCA CHAT Administrator (VCCA Member: 9802)
|
|
|
|
Joined: Jul 2005
Posts: 3,192 Likes: 8
ChatMaster - 3,000
|
ChatMaster - 3,000
Joined: Jul 2005
Posts: 3,192 Likes: 8 |
I was unable to connect to chat this morning again, 6.00 am AEDST. I did the standard windows connection check , and came back with the server is on line but not accepting connections.
JACK
|
|
|
|
Joined: Nov 2001
Posts: 5,906 Likes: 8
Former ChatMaster ChatMaster - 5,000
|
OP
Former ChatMaster ChatMaster - 5,000
Joined: Nov 2001
Posts: 5,906 Likes: 8 |
Okay... just spent the past 6 hours working on a new solution. Installed it 5 minutes ago and it appears to be working.
Please let me know if you experience any further issues over the next few days.
Email: webmaster@vcca.org
Thanks!!
Bill Barker Previous VCCA CHAT Administrator (VCCA Member: 9802)
|
|
|
|
Joined: Nov 2001
Posts: 5,906 Likes: 8
Former ChatMaster ChatMaster - 5,000
|
OP
Former ChatMaster ChatMaster - 5,000
Joined: Nov 2001
Posts: 5,906 Likes: 8 |
Members -- read the latest H-E-R-E
|
|
|
|
Joined: Apr 2006
Posts: 3,511 Likes: 48
ChatMaster - 3,000
|
ChatMaster - 3,000
Joined: Apr 2006
Posts: 3,511 Likes: 48 |
Bill, thanks for fighting the good fight!
Dean
Dean 'Rustoholic' Meltz old and ugly is beautiful!
|
|
|
|
Joined: Nov 2001
Posts: 29,863
Tech Advisor ChatMaster - 25,000
|
Tech Advisor ChatMaster - 25,000
Joined: Nov 2001
Posts: 29,863 |
The Mangy Old Mutt
"If It's Not Junk.....It's Not Treasure!"
|
|
|
|
Joined: Nov 2001
Posts: 5,906 Likes: 8
Former ChatMaster ChatMaster - 5,000
|
OP
Former ChatMaster ChatMaster - 5,000
Joined: Nov 2001
Posts: 5,906 Likes: 8 |
I'm going to post most of my comments in the Members area. I don't want to publicize what the current actions are.
Bill Barker Previous VCCA CHAT Administrator (VCCA Member: 9802)
|
|
|
|
Joined: Nov 2001
Posts: 29,863
Tech Advisor ChatMaster - 25,000
|
Tech Advisor ChatMaster - 25,000
Joined: Nov 2001
Posts: 29,863 |
The Mangy Old Mutt
"If It's Not Junk.....It's Not Treasure!"
|
|
|
|
Joined: Jul 2005
Posts: 3,192 Likes: 8
ChatMaster - 3,000
|
ChatMaster - 3,000
Joined: Jul 2005
Posts: 3,192 Likes: 8 |
Chatter works with lightning speed now !!, after not being able to get on at all this morning. 
JACK
|
|
|
|
Joined: Nov 2001
Posts: 5,906 Likes: 8
Former ChatMaster ChatMaster - 5,000
|
OP
Former ChatMaster ChatMaster - 5,000
Joined: Nov 2001
Posts: 5,906 Likes: 8 |
good.... I just made a break-through on a couple of things.
And I have blocked a couple of "good guys".... see member area for more info. :-)
Bill Barker Previous VCCA CHAT Administrator (VCCA Member: 9802)
|
|
|
|
Joined: Nov 2001
Posts: 29,863
Tech Advisor ChatMaster - 25,000
|
Tech Advisor ChatMaster - 25,000
Joined: Nov 2001
Posts: 29,863 |
The Mangy Old Mutt
"If It's Not Junk.....It's Not Treasure!"
|
|
|
|
Joined: Nov 2001
Posts: 5,906 Likes: 8
Former ChatMaster ChatMaster - 5,000
|
OP
Former ChatMaster ChatMaster - 5,000
Joined: Nov 2001
Posts: 5,906 Likes: 8 |
I see in the log files that 14 other people were using Chat online during that time.
I'm checking further.....
Bill Barker Previous VCCA CHAT Administrator (VCCA Member: 9802)
|
|
|
|
Joined: Nov 2001
Posts: 29,863
Tech Advisor ChatMaster - 25,000
|
Tech Advisor ChatMaster - 25,000
Joined: Nov 2001
Posts: 29,863 |
I tried to get on Chevy Chatter at least six times within that two hour period and it was a no-go each time. I could go to other talk sites, other web sites but not Chevy Chatter. I made a post on Chevy Chatter at 7:11 AM and then I clicked on "submit". From that time on and for the next two hours I was unable to access Chevy Chatter. Tried on another computer and it was the same there as well. 
The Mangy Old Mutt
"If It's Not Junk.....It's Not Treasure!"
|
|
|
|
Joined: Nov 2001
Posts: 5,906 Likes: 8
Former ChatMaster ChatMaster - 5,000
|
OP
Former ChatMaster ChatMaster - 5,000
Joined: Nov 2001
Posts: 5,906 Likes: 8 |
Looks like we've had pretty good performance since Sunday afternoon. I'm adding 3 or 4 IP's a day to the banned list. been watching the log files real close. Next slowdown will probably occur Sunday morning at 3 am PDT (USA West Coast). That's when the backup kicks in. I'll be watching this time. --Bill
|
|
|
|
Joined: Nov 2002
Posts: 7,472 Likes: 26
ChatMaster - 7,000
|
ChatMaster - 7,000
Joined: Nov 2002
Posts: 7,472 Likes: 26 |
I just logged in and it took about 3 minutes to connect and a couple more to log in. 6:32 pm to 6:38 eastern time.
Steve D
|
|
|
|
Joined: Nov 2001
Posts: 5,906 Likes: 8
Former ChatMaster ChatMaster - 5,000
|
OP
Former ChatMaster ChatMaster - 5,000
Joined: Nov 2001
Posts: 5,906 Likes: 8 |
The error and activity logs show someone logged in every minute for the hour before and after that. So, it "appears" that it was working.
But -- I also see about a thousand errors every 1 to 2 seconds looking for a non-existent file. Weird. Also... there was ONE bot trying to get in during this time. From Redmond Washington.
So... I'm more confused that before I began. but... I'll keep looking. At least it's working now.
Bill Barker Previous VCCA CHAT Administrator (VCCA Member: 9802)
|
|
|
|
Joined: Nov 2002
Posts: 7,472 Likes: 26
ChatMaster - 7,000
|
ChatMaster - 7,000
Joined: Nov 2002
Posts: 7,472 Likes: 26 |
I had trouble getting in to VCCA earlier, but all seems fine now.
Steve D
|
|
|
|
Joined: Nov 2001
Posts: 5,906 Likes: 8
Former ChatMaster ChatMaster - 5,000
|
OP
Former ChatMaster ChatMaster - 5,000
Joined: Nov 2001
Posts: 5,906 Likes: 8 |
Yes, I was actually "riding" the system earlier today when it crashed. There's even a possibility that I may have caused it but we aren't sure.
I was unable to restart it so I opened a trouble ticket. It took them 2 or 3 hours to fix it too. We still don't know WHAT caused it since the log files don't show any errors...
I'll be digging into them for the next hour or so...
Darn. This would have made 6 days in a row without any issues.
|
|
|
|
Joined: Nov 2002
Posts: 7,472 Likes: 26
ChatMaster - 7,000
|
ChatMaster - 7,000
Joined: Nov 2002
Posts: 7,472 Likes: 26 |
Also I am unable to view the links posted that oldie has viewed. I either get an error has occurred or no sound. Just need to know if it's on my end or not. Certainly appreciate the help.
Steve D
|
|
|
|
Joined: Nov 2001
Posts: 5,906 Likes: 8
Former ChatMaster ChatMaster - 5,000
|
OP
Former ChatMaster ChatMaster - 5,000
Joined: Nov 2001
Posts: 5,906 Likes: 8 |
Bill Barker Previous VCCA CHAT Administrator (VCCA Member: 9802)
|
|
|
|
Joined: Nov 2001
Posts: 29,863
Tech Advisor ChatMaster - 25,000
|
Tech Advisor ChatMaster - 25,000
Joined: Nov 2001
Posts: 29,863 |
The Mangy Old Mutt
"If It's Not Junk.....It's Not Treasure!"
|
|
|
|
Joined: Nov 2002
Posts: 7,472 Likes: 26
ChatMaster - 7,000
|
ChatMaster - 7,000
Joined: Nov 2002
Posts: 7,472 Likes: 26 |
My error. I can view the links, its just that once I follow them I have no sound and/or an error message. It's probably just a setting on my end. I'm sure you have enough to do battling the "gremlins".
Steve D
|
|
|
|
Joined: Nov 2001
Posts: 5,906 Likes: 8
Former ChatMaster ChatMaster - 5,000
|
OP
Former ChatMaster ChatMaster - 5,000
Joined: Nov 2001
Posts: 5,906 Likes: 8 |
Amazing... I was ready for last night's backup slowdown... and it didn't happen!!!
In anticipation of the usual Sunday 3 am slowdown (during the weekly backups), I set up a special program and tracked PINGS to the VCCA server every FOUR MINUTES all night long.
I stayed up until 1 am watching the service .... There was no problems... So I started the AUTO PING program. At 9 am I checked the log listing, and EVERY SINGLE PING was successfull -- every four minutes all night long. Ha!
Oh, well.... maybe next time.
Oh, yes... I did add another 3 addresses yesterday to our banned list. It does seem to be working.
--bb
|
|
|
|
Joined: Oct 2004
Posts: 535
Oil Can Mechanic
|
Oil Can Mechanic
Joined: Oct 2004
Posts: 535 |
Thank you for all your work on this!
|
|
|
|
Joined: Dec 2005
Posts: 113
Shade Tree Mechanic
|
Shade Tree Mechanic
Joined: Dec 2005
Posts: 113 |
Good morning Bill I've logged in several times this morning and every time I go to the Chat, the system logs me out and I an unable to post a topic for discussion. pottejo
|
|
|
|
|