Well at least we got a backup, right?
Right???
It last ran a week ago and we technically haven’t tested it. Just our hot replicas which also just deleted all that data.
And of course by now every downstream system replicated AND CACHED that data.
Holy shit the truth with replication deleting the data you needed too true lmao
Backup? What is this backup you speak of?
This is what we in the industry refer to as a “big oof.”
I thing the technical term for this is an RGE.
(Resume Generating Event)
8388409 = 2^23 - 199
I may have noticed this on a certain other aggregator site once upon a time, but I’m still none the wiser as to why.
199 rows kind of makes sense for whatever a legitimate query might have been, but if you’re going to make up a number, why 2^23? Why subtract? Am I metaphorically barking up the wrong tree?
Is this merely a mistyping of 8388608 and it was supposed to be ±1 row? Still the wrong (B-)tree?
WHY DO I CARE
Are you Ramanujam reborn or a nerd who put every number they found on wolfram alpha?
Ramanujan reborn - the main protagonist from the Wheel of Maths books.
In a place for programmer humour, you’ve got to expect there’s at least one person who knows their powers of two. (Though I am missing a few these days).
As for considering me to be Ramanujan reborn, if there’s any of Srinivasa in here, he’s not been given a full deck to work with this time around and that’s not very karmic of whichever deity or deities sent him back.
I know up to like 2^16 or maybe 2^17 while sufficiently caffeinated. Memorizing up to, or beyond, 2^23 is nerd award worthy.
I know that 2^20 is something more that a million because is the maximum number of rows excel can handle.
For me it’s: 2^1 to 2^16 (I remember the 8-bit era), a hazy gap and then 2^24 (the marketing for 24 bit colour in the 90s had 16777216 plastered all over it). Then it’s being uncomfortably lost up to 2^31 and 2^32, which I usually recognise when I see them (hello
INT_MAXandUINT_MAX), but I don’t know their digits well enough to repeat. 2^64 is similar. All others are incredibly vague or unknown.2^23 as half of 2^24 and having a lot of 8s in it seems to have put it into the “recognisable” category for me, even if it’s in that hazy gap.
So I grabbed a calculator to confirm.
It’s a good way to wake yourself up in the morning
Doctors HATE this one simple trick! Lose up to 100% of MyChart data - and KEEP it off!
Can help reduce blood pressure, high cholesterol, weight, height, gender, name and more to NULL! Wake up feeling NULL and NULL!
You can also do this by forgetting a WHERE clause. I know this because I ruined a production database in my early years.
Always write your where before your insert, kids.
Always start every command with EXPLAIN and don’t remove it until you’ve run it
I learned the same lesson the same way 😞
For everyone’s sanity, please restrict access to the prod DB to like two people. No company wants that to happen to them, and no developer wants to do that.
Me applying for any database access ever: “read only. I do not want write. READ ONLY.”
Just a funny story. All of our devs and even BAs used to have prod access. We all knew this was a bad idea and put in a process of hiring a DBA.
I think in the first two weeks the DBA screwed up prod twice. I can’t remember the first mess up but the second he had a lock on the database and then went to lunch.
We eventually hired two awesome DBAs to replace that one but oh boy.
Imagine being hired to help prevent people from fucking something up, only to fuck that thing up in your first week—not once, but twice. You’d think after the first time it wouldn’t happen again…
BEGIN
UPDATE
SELECT
COMMIT
That’s what backups are for.
Checking the backups… Ah yes, the backup done in August 2017.
Hello boss, I broke the company. I’ll see myself out
If you don’t have apt backups, that is a failure of the process, not yours.
Plot twist, they were also the one responsible for developing the backup process.
You should take it upon yourself to make regular backups in case you fuck up really bad. I had an intern that deleted everything on its fifth day, luckily l was automatically making backups two times a day, so it was fine.
Yep I do that on a local project basis before I make any updates. Saved me a couple times from my own mistakes 😅
Why would an intern be allowed anywhere near prod DB? Do you not have lower environments?
Company was a shitshow, new features or changes were expected immediately, so we got used to work directly on prod. I told him to test anything on a dummy DB and show me before we submit it, but he got around it when I wasn’t looking. The security tools were garbage, I wasn’t allowed to change permissions.
Sounds fun! As long as you have no stake in the company lol
I left to pursue my studies, the intern took my place and was put in charge of everything, I don’t know how he’s doing now and I don’t really care.
I’m dying lol
Looks like little bobby tables is at it again. (edit: for reference: https://xkcd.com/327/)
Edit #2: For lemmy app users: https://xkcd.com/327
And thanks to @[email protected] for the correction.
Rollback.
I don’t understand environments that don’t wrap things in transactions by default.
Especially since an update or delete without a where clause is considered valid.
No transaction to rollback
Ah reminds me of the time (back in the LAMP days) when I tried to apply this complicated equation that sales had come up with to our inventory database. This was one of those “just have the junior run it at midnight” type of shops. Anyway, I made a mistake and ended up exactly halving all inventory prices on production. See OP’s picture for my face.
In retrospect, I’m thankful for that memory.
oopsie daisy moment
'content':)Whenever SQL databases come up, I always think of this video from one of the greatest ‘content’ creators in history.
I actually screwed up twice on dev environment. Luckily the second case was salvageable without using data from an old backup(I wasn’t given one that time) and I managed to sweep it up fast.
I started testing my queries super carefully after the first incident, but I was too tired once that I forgot to restrict the update scope for testing and screwed up again.
