What in the fuck is this idiot doing? I’ve process datasets far larger than that and never once have I run into a hard drive “overheat”. I mean what level of incompetence do you have to have to get a hard drive to overheat processing a measley 60K rows of data?
I’ve met people who think their desktop PC is the “hard drive” but they’re all over 60.
This cannot be real, wtf. This is cartoon levels of ineptitude.
Or sabotage by someone heading out? Please let this be resistance sabotage they haven’t noticed yet.
You guys arent running your software off raspberry pi’s with sdcards from the gas station?
My allowance is 5$ a month!
Look, all I’m saying is give Pis a chance.
I think my Pi could process 60k rows without overheating. And the poor thing is dangling behind my bookshelf from its power cord with a fine layer of dust coating every inch of it.
Why would you do this to that poor rPi?
Electronics get what they deserve.
Maybe he left his shed lizard skin on top of the hard drive that caused the overheating?
When the only thing that is stopping kids from dismantling your government is an O(N^N) algorithm
Are you telling me there’s a difference between an inner and a cross join?
Cross join is obviously faster, I don’t even have to write “on”
60k rows of anything will be pulled into the file cache and do very little work on the drive. Possibly none after the first read.
You can put 60k rows in Excel 95.
Not if each row is pi!
Maybe
Maybe
They are just making shit up and doing jack shit
I really hope so.
They even doubled down: https://xcancel.com/DataRepublican/status/1900565922831618202#m
Well thanks for trying to keep us from catching it I guess
thank god, i am going to be ok, i was worried it might be contagious (it’s not, he just has brain damage)
Can you screenshot or something? I can’t load that link
Does this one work? https://nitter.net/DataRepublican/status/1900565922831618202#m Its a bit too long to screenshot.
That works, thanks
So it was a probably shitty external HD… that’s a whole other can of worms
I didn’t know hard drive overheating was a thing. Should I be worried that my 5 year old hard drive is about to overheat. I mean is this actually a floppy disk or something?
it is, in the select event that your platter bearing fails, in which case it would be very, very obvious.
So you’ve had this happen?
no. but generally spinning things that spin at several thousands of RPM that are spinning on a bearing, that no longer have a bearing usually sort of uh, tend to be VERY noisy.
Ok, but do you know anyone this has happened to? I don’t and I have some pretty old drives I still use. I tend to just buy more. Also most drives these days are solid state aren’t they? This just feels like a low probability event to me. Overheating RAM or the CPU or GPU, sure, but hard drive?
not personally, i may have seen a video or two of it happening, but it’s hard to tell whether the head is dragging against the platter, or it’s the bearing, either one of those makes horrendous noise.
If you’re worried about it happening on a drive you own, you should copy that data somewhere else as a backup, ideally sooner rather than later. If you’re curious about the health of the drive you do stuff like SMART tests as well.
Yeah, most drives are solid state now, unless you’re buying hdds for archival purposes, still cheaper and denser in most cases. It’s a low probability failure, until the drive meets EOL, in which case it’s a mechanical wear part, either the motor or the bearing fails. One of them will fail first, probably the bearing.
The bearing failing would likely result in the HDD overheating as a result. Assuming the platter still spins, but that’s the only scenario i can think of where that would happen, unless you dump a very specific amount of continuous current into the read arm coils. That might also cause it, but it’s not likely at all.
An ssd “overheating” is more likely, but it shouldn’t cause too many issues, maybe premature degradation over long term use, and slowing of read/write speeds, or in some cases, an improvement, but other than that it should be business as normal. You would have to hit it with like a heat gun, to get a hardware failure or something like that.
I’m not actually worried about my drives. In fact that was kind of the point. I was kidding around because this excuse that the hard drive overheated sounds a little like a car running out of blinker fluid.
hard disk drives do still exist, and are useful for some stuff. mostly they’re cheaper. I think better for stuff you write to often?
SSD’s can overheat. but, again, there are usually sensors to throttle them when you’re in danger of this, and this idiot probably disabled those. because safety is for cucks i guess.
I had it happen to a random hard drive I bought, old bastard found in a bin at a local thrift store. Anyways had a big dent in it and while it started up, even booting into windows XP it cooked itself made a screeching noise and was hot to the touch. Don’t think I need to explain that the big dent was probably the source of the overheating, anyways had to use an oven mitt to relocate it to a metal bucket of water where it boiled for a minute.
So the story is plausible then?
Sure if they physically damaged the hard drive, though at that point the failure and the code were largely unrelated. Though if they were using a computer with an SSD only I would have no clue. Assuming they aren’t lying id have to look at the computer physically, perhaps even pry it open to see if there’s any noticable damage.
There’s also the possibility that they are just stupid and overheated their CPU by having too many apps open. Then they blamed their hard drive overheating because they are stupid.
it is a thing, but any competently designed computer should have things in place to prevent this.
unless you’re an arrogant dipshit and disable all the hardware safeties on your computer to make it go faster and wear harder.
When an HDD works continuously it can heat up to above 60 °C if proper air circulation is not allowed, which can cause a very premature failure. In fact, it should be kept under 40 °C to achieve the intended lifespan. Unfortunately, PC cases are usually not great at removing heat from the HDD by default.
As for your drive, it most likely has a temperature sensor so it can be displayed by various utilities.
I have a 12v fan running at 5v spitting air on my hdds, and that’s enough for them to go from 55°C to 29°C, lol.
I did that too, the tiny fan is pretty much silent at 5V. The HDD has so much surface area it only needs a little air circulation.
60k isn’t that much, I frequently run scripts against multiple hundreds of thousands at work. Wtf is he doing? Did he duplicate the government database onto his 2015 MacBook Air?
I’ve run searches over 60k lines of raw JSON on a 2015 MacBook air without any problems.
No, its an external drive, appearently.
60k is laughably, embarrassingly small. It’s still sqlite-sized.
Sqlite can easily handle millions of rows. Don’t sell it short
How about a 6.4TB sqlite database?
Should be enough to hold 60k rows
I’m not
I have an sqlite db that is a few GB in size, game saves using the format. Sadly almost all blob data, would love to play with it if it was a bit more readable
60k is single json file
i mean its even excel sized depending on how many columns. This is seriously sad and alarming
Hey now that’s real close to the 65,535 16-bit limit (from 20 years ago)
Holy shit if this ids lm issue that’s too funny
I’d do that if I was given so much stupid access
deleted by creator
A TI-86 can query 60k rows without breaking a sweat.
If his hard drive overheated from that, he is doing something very wrong, very unhygienic, or both.
What? You don’t run your hard drives in the oven while baking brownies? It makes them zesty.
There must be more join statements than column names
He probably mining crypto on top of running his SQL queries.
I mean if we were to sort of steelman this thing, there sure can be database relations and queries that hit only 60k rows but are still hteavy as fuck.
Don’t know what Elmos minions are doing, but I’ve written code at least equally unefficient. It was quite a few years ago (the code was in written in perl) and I at least want to think that I’m better now (but I’m not paid to code anymore). The task was to pull in data from a CSV (or something like that, as I mentioned, it’s been a while) and it needed conversion to XML (or something similar).
The idea behind my code was that you could just configure which fields you want from arbitary source data and on where to place them on the whatever supported destination format. I still think that the basic idea behind that project is pretty neat, just throw in whatever you happen to have and have something completely else out of the other end. And it worked as it should. It was just stupidly hungry for memory. 20k entries would eat up several gigabytes of memory from a workstation (and back then it was premium to have even 16G around) and it was also freaking slow to run (like 0.2 - 0.5 seconds per entry).
But even then I didn’t need to tweet that my hard drive is overheating. I well understood that my code is just bad and I even improved it a bit here and there, but it was still so very slow and used ridiculous amounts of RAM. The project was pretty neat and when you had few hundred items to process at a time it was even pretty good, there was companies who relied on that code and paid for support. It just totally broke down with even a slightly bigger datasets.
But, as I already mentioned, my hard drive didn’t overheat on that load.
Seriously - I can parse multiple tables of 5+ million row each… in EXCEL… on a 10 year old desktop and not have the fan even speed up. Even the legacy Access database I work with handles multiple million+ row tables better than that.
Sounds like the kid was running his AI hamsters too hard and they died of exhaustion.
Excel have a limit of 2^20 rows, something more that 1M. Curious what version of excel are you using for that.
You’re correct - the standard tabs can only hold roughly 1.2 million rows.
The way to get around that limitation is to use the Data Model within Power Pivot:
It can accept all of the data connections a standard Power Query can (ODBC, Sharepoint, Access, etc):
You build the connection in Power Pivot to your big tables and it will pull in a preview. If needed, you can build relationship between tables with the Relationship Manager. You can also use DAX to build formulas just like in a regular Excel tab (very similar to Visual Basic). You can then run Pivot Tables and charts against the Data Model to pull out the subsets of data you want to look at.
The load times are pretty decent - usually it takes 2-3 minutes to pull a table of 4 million rows from an SQL database over ODBC, but your results may vary depending on datasource. It can get memory intensive, so I recommend a machine with a decent amount of RAM if you’re going to build anything for professional use.
The nice thing about building it out this way (as opposed to using independent Power Queries to bring out your data subsets) is that it’s a one-button refresh, with most of the logic and formulas hidden back within the Data Model, so it’s a nice way to build reports for end-users that’s harder for them to fuck up by deleting a formula or hiding a column.
Oh yes, I remember using power query for a few months once I started working with bigger databases, but I saw that moving to Python would be better carrer wise and never came back to excel to do actual work (but at the end everything get exported to excel)
I smell something, but it’s not overheating electronics.
I’ve processed over 5 million records on a laptop that’s almost 10 years old. it took two days to get my results.
there’s no way 60,000 records overheated ANYTHING.
Doesn’t actually say that 60k overheated his drive. He says that he ran a run on 60k, and that he couldn’t do the whole database due to overheating. Two unrelated statements except that 60k is the lower bound for what he could process.
Doesn’t mean he knows what he’s doing though, as pretty huge datasets are processable on quite modest hardware if you do it right.
that’s somehow worse.
a “data analyst” couldn’t cut up the work into a parallel processes and run them synchronously? what the actual fuck?
“sorry, I can only do 60k at a time.”
just fucking split them up into 6 parallel batch processes running 10k at a time. it’s fucking math, not rocket science. I’m not even an analyst and I could fucking do that much.
I don’t want to take away from the valid point of your comment, but rocket science is almost exclusively math
“I store my records on vinyl. You’ve probably never heard of them.”
Hard drive? How old is his computer?
Unless I’m misreading it which is possible it’s awfully late, he said he processed 60,000 rows didn’t find what he was looking for but his hard drive overheated on the full pass.
Discs don’t overheat because there was load. Even if he f***** up and didn’t index the data correctly (I assume it’s a relational database since he’s talking about rows) The disc isn’t just going to overheat because the job is big. It’s going to be lack of air flow or lack of heatsink.
I guarantee you he was running on an external NVMe, and one of those little shitty-ass Chinese enclosures. Or maybe one of those self immolating SanDisk enclosures. Hell, maybe he’s on a desktop and he slept a raw NVMe on his motherboard without a heatsink
There are times when you want a brilliant college student on your team, But you need seasoned professionals to help them through the things they’ve never seen before and never done before.
Somehow I feel over clicking without understanding of the consequences sounds like something a techbro would do
music theaters also have rows, and they run on sql so logic checks out.
Can’t be a relational database, Musk said the government doesn’t use SQL.
Lol he also said cybertrucks don’t suck ;)
grep would eat 60k lines without a problem too.
He said many things.
Please remember that he is a genius. Only geniuses say a lot of things.
I rest my case.
yes but also why say 60K when you could have literally said anything? I mean surely the fact that he thinks 60K rows a big number is already explaining alot lol.
It’s bait.
They probably have an explanation tweet at the ready to make more sense of it. They just want enough 'hurr durr these idiot" comments before they reverse Uno card this with more context.
Based on all that has been going on, I feel like they don’t really have the capacity to think more than one step ahead. They do sth stupid and then they usually follow up with “lol joke” or “lol you can’t understand”
There are times when you want a brilliant college student on your team, But you need seasoned professionals to help them through the things they’ve never seen before and never done before.
Honestly, any sweet, white-haired old lady who keeps pictures of her dogs and grandkids on her desk who’s been doing data entry for 15 years could do circles around these clowns.
But she might also have the wisdom and perception to know we’re not supposed to be doing this “work” at all, which is why he recruits naive teenagers and college kids who are still emotionally immature to think that this is going to be their “destiny” or their opportunity to get into the big leagues of business.
I keep hearing things about these hires he has, I don’t think they’re naive, At least not as such. They seem to be more power hungry trust fund babies.
But yeah, people with a few years in them would be a moral liability in that line of work.
Yeah if you read more of these guys tweets they are clearly in politics. One message tried to claim trump loves kids (to be clear: in the abstract sense, not in the he definitely fucked kids on an island with Epstein sense). Then they tried to twist the words to say “why don’t you love kids”. It was clumsy like you’d expect from someone who is practically a teenager, but the core is an attempt to follow the usual right wing playbook.
Molly White is very bright, and she makes them feel inadequate so they “have to” attack her. It’s truly pathetic.
But her last name is White so it’s a real dilemma for them.
“YOU’RE JUST JEALOUS” is such a fucking pussy-ass response, too.
heh
Literally every time someone dismisses Wikipedia, it’s because they believe something crazy that Wikipedia told them is wrong.
I checked conservapedia once, and its actually unhinged. If someone tells you to look at that, or reccommends it, they’re crazy.
Did they ever finish their own bible translation? The one they started because King James was too woke.
YES . its so unhinged . they have an entire page discussing if Obama is actually a Muslim
Flashback to 2014.
“I read a book with a typo once. Libraries are a scam.”
Libraries are a scam if they weren’t they’d still have VHS rentals.
God they picked out the ONE possible thing they could criticize her for, there’s like 3 other things RIGHT NEXT TO THAT
“Software Engineer” was literally right next to it.
She is vehemently against crypto. She runs the newsletter “web3 is going great” - https://www.web3isgoinggreat.com/
The trick is to take the hard drive out of the EVA foam.
Instructions unclear, dick stuck in GPU.
