Stevo985 Posted June 29, 2013 VT Supporter Share Posted June 29, 2013 In my last job I was the DBA for a 4TB database of a rolling last 30 days of 6 million insurance quotes a day (it was the back end web service for all the price comparison sites to get their quotes from). It was a circular replicated master-master cluster so any queries other than selects will get run the same on the other server. There was some corruption in one of the nodes' copy of the database so I had the bright idea to just direct all traffic to the good node and run a mysqldump on the bad node pulling good data from the good node. The first bit a mysqldump does is DROP TABLE.. which then gets replicated to the other node.. Database screwed while 300 quotes a second are coming in. Worst bit was there was no backup as the RTO is so long as to be unworkable. So no one could retrieve any quotes from any of the places like confused, gocompare etc because I screwed the database. It gradually filled back up obviously. I won't do that again! Sent from my Galaxy Nexus using Tapatalk 4 Beta This may as well be written in latin. 1 Link to comment Share on other sites More sharing options...
chrisp65 Posted June 29, 2013 Share Posted June 29, 2013 In my last job I was the DBA for a 4TB database of a rolling last 30 days of 6 million insurance quotes a day (it was the back end web service for all the price comparison sites to get their quotes from). It was a circular replicated master-master cluster so any queries other than selects will get run the same on the other server. There was some corruption in one of the nodes' copy of the database so I had the bright idea to just direct all traffic to the good node and run a mysqldump on the bad node pulling good data from the good node. The first bit a mysqldump does is DROP TABLE.. which then gets replicated to the other node.. Database screwed while 300 quotes a second are coming in. Worst bit was there was no backup as the RTO is so long as to be unworkable. So no one could retrieve any quotes from any of the places like confused, gocompare etc because I screwed the database. It gradually filled back up obviously. I won't do that again! Sent from my Galaxy Nexus using Tapatalk 4 Beta This may as well be written in latin. 11101100101001100100111110101110100010100100011101010 10100011001000100011001001001010100001011001000110010 1 Link to comment Share on other sites More sharing options...
8pints Posted June 29, 2013 Share Posted June 29, 2013 11101100101001100100111110101110100010100100011101010 10100011001000100011001001001010100001011001000110010 Now I understand. Link to comment Share on other sites More sharing options...
chrisp65 Posted June 29, 2013 Share Posted June 29, 2013 A couple of years back I was working as part of a consultant team and at a meeting where I wasn't present, the Project Manager claimed a design error was my fault in front of our mutual Client. It wasn't my fault and he already knew and accepted that, but for short term face saving on his part he let me take a fall without me being able to defend myself. I got to here about this, so I phoned him: brrrrr brrrr brrrrrr Him: Hi you've reached the VM of so and so, please leave a message. Me: Hi, yes there appears to have been a problem at the meeting, looks like the Client was mis informed on a couple of things and I'd like the opportunity to chat asap, thanks. then I clicked the phone off with the little red phone symbol and for my own benefit over the hands free continued along the lines of..... Me: turns out your a sneaky **** of a lying **** of snake that's going to have his head ripped off and **** in you useless lying **** I'm gonna **** destroy your worthless lying life you ****...... this continued right up to the point I heard his auto message say the voicemail was now full! Odd thing was, he never mentioned it and I never had a problem with him again. 3 Link to comment Share on other sites More sharing options...
Genie Posted June 29, 2013 Share Posted June 29, 2013 (edited) Not really a blunder but next week I will be falling out with my boss, and his boss big time as I expose them for **** up my performance review... which will probably mean theirs is shit. Edited June 29, 2013 by Tamuff_Villa Link to comment Share on other sites More sharing options...
Baselayers Posted June 29, 2013 Share Posted June 29, 2013 Nearly broke my neck and spine a few times. Harsh times! Link to comment Share on other sites More sharing options...
Anthony Posted June 29, 2013 VT Supporter Share Posted June 29, 2013 Yeah, um Jonesy, you know you're asking for music and that for your Olympic runs? Maybe you should just have Brahms or some gentle Sibelius? Don't want you going and breaking stuff again you know. Apart from records. You can do that. But only if you come to the Holte car park with your medals. Link to comment Share on other sites More sharing options...
maqroll Posted June 30, 2013 Share Posted June 30, 2013 Took on 4 drunk and beligerent guys as we we closing the bar....dumb move, but somehow I didn't get hurt. Link to comment Share on other sites More sharing options...
Seat68 Posted June 30, 2013 Share Posted June 30, 2013 Ages ago I had a job as a security guard, night shifts watching over a yard and office building, may get one driver coming in at 2am to drop off a truck but that was it. My time was spent watching telly and reading, and and rooting through offices, as I am nosey. So one driver would come in and because I was lazy, never signed them in, just let them drop their shit off. So turned out this driver was coming in, making nuisance phonecalls to a woman, and coming out with some pretty graphic and potentially disturbing shit. So, I had never signed this guy in and so for a month my employers assumed that I had been making phone calls to a random woman in Birmingham, could not prove it wasnt me, and at the disciplinary I left with the parting shot of go **** yourselves. Wonder now, over 15 years later if the driver was found out. Link to comment Share on other sites More sharing options...
jadenb42 Posted June 30, 2013 Share Posted June 30, 2013 I'm a sign maker, it's all road signs talking like the ones you seeing being plastered around the city centre shortly, make the big mofo ones for motorway... Anyway I had been doing mainly signs in aluminium all day, get this one job has about 40 decent size signs to do. Get it all cut in aluminium, thinking Yea glad that's done, the next department come along ready to lay the facings on and ask where the job is, I'm like there. They then point out its meant to be cut in a different material and much cheaper material called GRP :-( Link to comment Share on other sites More sharing options...
islingtonclaret Posted July 1, 2013 Share Posted July 1, 2013 In my last job I was the DBA for a 4TB database of a rolling last 30 days of 6 million insurance quotes a day (it was the back end web service for all the price comparison sites to get their quotes from). It was a circular replicated master-master cluster so any queries other than selects will get run the same on the other server. There was some corruption in one of the nodes' copy of the database so I had the bright idea to just direct all traffic to the good node and run a mysqldump on the bad node pulling good data from the good node. The first bit a mysqldump does is DROP TABLE.. which then gets replicated to the other node.. Database screwed while 300 quotes a second are coming in. Worst bit was there was no backup as the RTO is so long as to be unworkable. So no one could retrieve any quotes from any of the places like confused, gocompare etc because I screwed the database. It gradually filled back up obviously. I won't do that again! Can I reword your post for you? "I was in charge of a big database that handled quotes for comparison websites. It broke a bit, and I decided to fix it but completely busted it." Use of master-master replication is fine for talking to clients and developers, but I think you need to know your general audience; in this case a load of football fans. I can forgive the real terminology of nodes, replication etc. but in our line of work (which will be similar to yours) the use of "RTO" is regarded as "BusinessSpeak 2.0-prat-talk" 1 Link to comment Share on other sites More sharing options...
drat01 Posted July 1, 2013 Share Posted July 1, 2013 I disagree re RTO - that is still a key phrase in that statement. :-) Link to comment Share on other sites More sharing options...
tonyh29 Posted July 1, 2013 Share Posted July 1, 2013 think the worse blunder I ever made at work was getting a colleague a coffee and forgetting to put the sugar in it !! either I'm just brilliant or I've never really been given a job involving any responsibility ...... Link to comment Share on other sites More sharing options...
mjmooney Posted July 1, 2013 VT Supporter Share Posted July 1, 2013 For salaried employees, it seems to be virtually impossible to get sacked by a university. I've been trying for years, with no success. Link to comment Share on other sites More sharing options...
islingtonclaret Posted July 1, 2013 Share Posted July 1, 2013 I disagree re RTO - that is still a key phrase in that statement. :-) As far as I'm concerned, RTO is actually "recovery". There's nothing "objective" about it at all. Please see: http://www.plainenglish.co.uk/ Link to comment Share on other sites More sharing options...
StefanAVFC Posted July 1, 2013 Share Posted July 1, 2013 dunno if it constituents a cock-up but it was pretty embarrassing. In 2011, I was working at a summer camp on Long Island in New York and I'd just come in from a night off. When I came in I'd check to see all the kids were there and then go to bed. My co-counselor was dreadful at his job and would disappear when he was supposed to be on so i had to check these things. I came in, checked all the beds and one bed was empty. No biggie, probably just gone to the toilet. Waited like 10 minutes and went up to the toilet and he wasn't there. Bear in mind I was absolutely shattered from working crazy hours and hardly getting any sleep cause it was like 35 degrees and 90-100% humidity so I woke my co-counselor up and was like 'Kai isn't in his bed'. I thought I reacted quite calmly and just got my supervisor who was in the cabin opposite me and then it snowballed. The whole camp was woken up and we had a staff meeting. Went into the cabin again and he had sleepwalked and got into another kids' bed by mistake. Such a preventable situation and I got so much stick for it. But it went through both my supervisor and my so counselor, but they were waterfront so they could do no wrong. I **** hated that job. I worked proper hard all summer and so many of the other counselors slacked, got drunk/high when they were on etc but because I did arts instead of sport or waterfront, I wasn't respected cause our department was classed as a bit of a joke. Link to comment Share on other sites More sharing options...
villaajax Posted July 1, 2013 Share Posted July 1, 2013 think the worse blunder I ever made at work was getting a colleague a coffee and forgetting to put the sugar in it !! either I'm just brilliant or I've never really been given a job involving any responsibility ...... That's quite a mistake for a tea boy though. 1 Link to comment Share on other sites More sharing options...
coda Posted July 1, 2013 Share Posted July 1, 2013 dunno if it constituents a cock-up but it was pretty embarrassing. In 2011, I was working at a summer camp on Long Island in New York and I'd just come in from a night off. When I came in I'd check to see all the kids were there and then go to bed. My co-counselor was dreadful at his job and would disappear when he was supposed to be on so i had to check these things. I came in, checked all the beds and one bed was empty. No biggie, probably just gone to the toilet. Waited like 10 minutes and went up to the toilet and he wasn't there. Bear in mind I was absolutely shattered from working crazy hours and hardly getting any sleep cause it was like 35 degrees and 90-100% humidity so I woke my co-counselor up and was like 'Kai isn't in his bed'. I thought I reacted quite calmly and just got my supervisor who was in the cabin opposite me and then it snowballed. The whole camp was woken up and we had a staff meeting. Went into the cabin again and he had sleepwalked and got into another kids' bed by mistake. Such a preventable situation and I got so much stick for it. But it went through both my supervisor and my so counselor, but they were waterfront so they could do no wrong. I **** hated that job. I worked proper hard all summer and so many of the other counselors slacked, got drunk/high when they were on etc but because I did arts instead of sport or waterfront, I wasn't respected cause our department was classed as a bit of a joke. I worked in the kitchen on a camp and all the counselors wanted to be kitchen staff, all the kitchen staff wanted to be counselors. I couldn't be working with brattish kids. They threatened to send me back to Blighty one day as I refused to work my hangover was so bad. They didn't go through with their threat. Link to comment Share on other sites More sharing options...
StefanAVFC Posted July 1, 2013 Share Posted July 1, 2013 Summer camps in the whole are split between 2 groups of people. Who who are there to work and people who are there for a summer in America. Where I was last year was great. We got the balance between work and having fun spot on and everyone was on the same wavelength. Link to comment Share on other sites More sharing options...
drat01 Posted July 1, 2013 Share Posted July 1, 2013 I disagree re RTO - that is still a key phrase in that statement. :-) As far as I'm concerned, RTO is actually "recovery". There's nothing "objective" about it at all. Please see: http://www.plainenglish.co.uk/ I know what RTO means - I bloody should ! Link to comment Share on other sites More sharing options...
Recommended Posts