• inclementimmigrant@lemmy.world
    link
    fedilink
    arrow-up
    11
    ·
    edit-2
    22 hours ago

    What in the fuck is this idiot doing? I’ve process datasets far larger than that and never once have I run into a hard drive “overheat”. I mean what level of incompetence do you have to have to get a hard drive to overheat processing a measley 60K rows of data?

  • LillyPip@lemmy.ca
    link
    fedilink
    arrow-up
    92
    ·
    19 hours ago

    This cannot be real, wtf. This is cartoon levels of ineptitude.

    Or sabotage by someone heading out? Please let this be resistance sabotage they haven’t noticed yet.

    • wise_pancake@lemmy.ca
      link
      fedilink
      arrow-up
      6
      ·
      19 hours ago

      Are you telling me there’s a difference between an inner and a cross join?

      Cross join is obviously faster, I don’t even have to write “on”

  • jkercher@programming.dev
    link
    fedilink
    English
    arrow-up
    37
    ·
    21 hours ago

    60k rows of anything will be pulled into the file cache and do very little work on the drive. Possibly none after the first read.

  • Professorozone@lemmy.world
    link
    fedilink
    arrow-up
    17
    arrow-down
    1
    ·
    20 hours ago

    I didn’t know hard drive overheating was a thing. Should I be worried that my 5 year old hard drive is about to overheat. I mean is this actually a floppy disk or something?

        • KillingTimeItself@lemmy.dbzer0.com
          link
          fedilink
          English
          arrow-up
          14
          ·
          20 hours ago

          no. but generally spinning things that spin at several thousands of RPM that are spinning on a bearing, that no longer have a bearing usually sort of uh, tend to be VERY noisy.

          • Professorozone@lemmy.world
            link
            fedilink
            arrow-up
            3
            ·
            20 hours ago

            Ok, but do you know anyone this has happened to? I don’t and I have some pretty old drives I still use. I tend to just buy more. Also most drives these days are solid state aren’t they? This just feels like a low probability event to me. Overheating RAM or the CPU or GPU, sure, but hard drive?

            • KillingTimeItself@lemmy.dbzer0.com
              link
              fedilink
              English
              arrow-up
              1
              ·
              19 hours ago

              not personally, i may have seen a video or two of it happening, but it’s hard to tell whether the head is dragging against the platter, or it’s the bearing, either one of those makes horrendous noise.

              If you’re worried about it happening on a drive you own, you should copy that data somewhere else as a backup, ideally sooner rather than later. If you’re curious about the health of the drive you do stuff like SMART tests as well.

              Yeah, most drives are solid state now, unless you’re buying hdds for archival purposes, still cheaper and denser in most cases. It’s a low probability failure, until the drive meets EOL, in which case it’s a mechanical wear part, either the motor or the bearing fails. One of them will fail first, probably the bearing.

              The bearing failing would likely result in the HDD overheating as a result. Assuming the platter still spins, but that’s the only scenario i can think of where that would happen, unless you dump a very specific amount of continuous current into the read arm coils. That might also cause it, but it’s not likely at all.

              An ssd “overheating” is more likely, but it shouldn’t cause too many issues, maybe premature degradation over long term use, and slowing of read/write speeds, or in some cases, an improvement, but other than that it should be business as normal. You would have to hit it with like a heat gun, to get a hardware failure or something like that.

              • Professorozone@lemmy.world
                link
                fedilink
                arrow-up
                3
                ·
                19 hours ago

                I’m not actually worried about my drives. In fact that was kind of the point. I was kidding around because this excuse that the hard drive overheated sounds a little like a car running out of blinker fluid.

            • melpomenesclevage@lemmy.dbzer0.com
              link
              fedilink
              English
              arrow-up
              2
              ·
              18 hours ago

              hard disk drives do still exist, and are useful for some stuff. mostly they’re cheaper. I think better for stuff you write to often?

              SSD’s can overheat. but, again, there are usually sensors to throttle them when you’re in danger of this, and this idiot probably disabled those. because safety is for cucks i guess.

            • vaultdweller013@sh.itjust.works
              link
              fedilink
              arrow-up
              1
              ·
              19 hours ago

              I had it happen to a random hard drive I bought, old bastard found in a bin at a local thrift store. Anyways had a big dent in it and while it started up, even booting into windows XP it cooked itself made a screeching noise and was hot to the touch. Don’t think I need to explain that the big dent was probably the source of the overheating, anyways had to use an oven mitt to relocate it to a metal bucket of water where it boiled for a minute.

                • vaultdweller013@sh.itjust.works
                  link
                  fedilink
                  arrow-up
                  1
                  ·
                  19 hours ago

                  Sure if they physically damaged the hard drive, though at that point the failure and the code were largely unrelated. Though if they were using a computer with an SSD only I would have no clue. Assuming they aren’t lying id have to look at the computer physically, perhaps even pry it open to see if there’s any noticable damage.

                  There’s also the possibility that they are just stupid and overheated their CPU by having too many apps open. Then they blamed their hard drive overheating because they are stupid.

    • melpomenesclevage@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      13
      ·
      18 hours ago

      it is a thing, but any competently designed computer should have things in place to prevent this.

      unless you’re an arrogant dipshit and disable all the hardware safeties on your computer to make it go faster and wear harder.

    • ChaoticNeutralCzech@feddit.org
      link
      fedilink
      English
      arrow-up
      10
      ·
      edit-2
      19 hours ago

      When an HDD works continuously it can heat up to above 60 °C if proper air circulation is not allowed, which can cause a very premature failure. In fact, it should be kept under 40 °C to achieve the intended lifespan. Unfortunately, PC cases are usually not great at removing heat from the HDD by default.

      As for your drive, it most likely has a temperature sensor so it can be displayed by various utilities.

      • Estebiu@lemmy.dbzer0.com
        link
        fedilink
        arrow-up
        3
        ·
        12 hours ago

        I have a 12v fan running at 5v spitting air on my hdds, and that’s enough for them to go from 55°C to 29°C, lol.

        • ChaoticNeutralCzech@feddit.org
          link
          fedilink
          English
          arrow-up
          3
          ·
          edit-2
          10 hours ago

          I did that too, the tiny fan is pretty much silent at 5V. The HDD has so much surface area it only needs a little air circulation.

  • Tiefling IRL@lemmy.blahaj.zone
    link
    fedilink
    arrow-up
    141
    ·
    1 day ago

    60k isn’t that much, I frequently run scripts against multiple hundreds of thousands at work. Wtf is he doing? Did he duplicate the government database onto his 2015 MacBook Air?

    • socsa@piefed.social
      link
      fedilink
      English
      arrow-up
      9
      ·
      1 day ago

      I’ve run searches over 60k lines of raw JSON on a 2015 MacBook air without any problems.

    • 4am@lemm.ee
      link
      fedilink
      arrow-up
      74
      ·
      1 day ago

      A TI-86 can query 60k rows without breaking a sweat.

      If his hard drive overheated from that, he is doing something very wrong, very unhygienic, or both.

    • vga@sopuli.xyz
      link
      fedilink
      arrow-up
      1
      ·
      edit-2
      1 day ago

      I mean if we were to sort of steelman this thing, there sure can be database relations and queries that hit only 60k rows but are still hteavy as fuck.

    • IsoKiero@sopuli.xyz
      link
      fedilink
      English
      arrow-up
      13
      arrow-down
      1
      ·
      edit-2
      1 day ago

      Don’t know what Elmos minions are doing, but I’ve written code at least equally unefficient. It was quite a few years ago (the code was in written in perl) and I at least want to think that I’m better now (but I’m not paid to code anymore). The task was to pull in data from a CSV (or something like that, as I mentioned, it’s been a while) and it needed conversion to XML (or something similar).

      The idea behind my code was that you could just configure which fields you want from arbitary source data and on where to place them on the whatever supported destination format. I still think that the basic idea behind that project is pretty neat, just throw in whatever you happen to have and have something completely else out of the other end. And it worked as it should. It was just stupidly hungry for memory. 20k entries would eat up several gigabytes of memory from a workstation (and back then it was premium to have even 16G around) and it was also freaking slow to run (like 0.2 - 0.5 seconds per entry).

      But even then I didn’t need to tweet that my hard drive is overheating. I well understood that my code is just bad and I even improved it a bit here and there, but it was still so very slow and used ridiculous amounts of RAM. The project was pretty neat and when you had few hundred items to process at a time it was even pretty good, there was companies who relied on that code and paid for support. It just totally broke down with even a slightly bigger datasets.

      But, as I already mentioned, my hard drive didn’t overheat on that load.

    • arotrios@lemmy.world
      link
      fedilink
      English
      arrow-up
      9
      ·
      1 day ago

      Seriously - I can parse multiple tables of 5+ million row each… in EXCEL… on a 10 year old desktop and not have the fan even speed up. Even the legacy Access database I work with handles multiple million+ row tables better than that.

      Sounds like the kid was running his AI hamsters too hard and they died of exhaustion.

        • arotrios@lemmy.world
          link
          fedilink
          English
          arrow-up
          5
          ·
          24 hours ago

          You’re correct - the standard tabs can only hold roughly 1.2 million rows.

          The way to get around that limitation is to use the Data Model within Power Pivot:

          It can accept all of the data connections a standard Power Query can (ODBC, Sharepoint, Access, etc):

          You build the connection in Power Pivot to your big tables and it will pull in a preview. If needed, you can build relationship between tables with the Relationship Manager. You can also use DAX to build formulas just like in a regular Excel tab (very similar to Visual Basic). You can then run Pivot Tables and charts against the Data Model to pull out the subsets of data you want to look at.

          The load times are pretty decent - usually it takes 2-3 minutes to pull a table of 4 million rows from an SQL database over ODBC, but your results may vary depending on datasource. It can get memory intensive, so I recommend a machine with a decent amount of RAM if you’re going to build anything for professional use.

          The nice thing about building it out this way (as opposed to using independent Power Queries to bring out your data subsets) is that it’s a one-button refresh, with most of the logic and formulas hidden back within the Data Model, so it’s a nice way to build reports for end-users that’s harder for them to fuck up by deleting a formula or hiding a column.

          • driving_crooner@lemmy.eco.br
            link
            fedilink
            arrow-up
            4
            ·
            22 hours ago

            Oh yes, I remember using power query for a few months once I started working with bigger databases, but I saw that moving to Python would be better carrer wise and never came back to excel to do actual work (but at the end everything get exported to excel)

  • GreenKnight23@lemmy.world
    link
    fedilink
    arrow-up
    60
    ·
    22 hours ago

    I smell something, but it’s not overheating electronics.

    I’ve processed over 5 million records on a laptop that’s almost 10 years old. it took two days to get my results.

    there’s no way 60,000 records overheated ANYTHING.

    • wewbull@feddit.uk
      link
      fedilink
      English
      arrow-up
      23
      arrow-down
      2
      ·
      21 hours ago

      Doesn’t actually say that 60k overheated his drive. He says that he ran a run on 60k, and that he couldn’t do the whole database due to overheating. Two unrelated statements except that 60k is the lower bound for what he could process.

      Doesn’t mean he knows what he’s doing though, as pretty huge datasets are processable on quite modest hardware if you do it right.

      • GreenKnight23@lemmy.world
        link
        fedilink
        arrow-up
        12
        arrow-down
        1
        ·
        18 hours ago

        that’s somehow worse.

        a “data analyst” couldn’t cut up the work into a parallel processes and run them synchronously? what the actual fuck?

        “sorry, I can only do 60k at a time.”

        just fucking split them up into 6 parallel batch processes running 10k at a time. it’s fucking math, not rocket science. I’m not even an analyst and I could fucking do that much.

  • rumba@lemmy.zip
    link
    fedilink
    English
    arrow-up
    123
    ·
    18 hours ago

    Unless I’m misreading it which is possible it’s awfully late, he said he processed 60,000 rows didn’t find what he was looking for but his hard drive overheated on the full pass.

    Discs don’t overheat because there was load. Even if he f***** up and didn’t index the data correctly (I assume it’s a relational database since he’s talking about rows) The disc isn’t just going to overheat because the job is big. It’s going to be lack of air flow or lack of heatsink.

    I guarantee you he was running on an external NVMe, and one of those little shitty-ass Chinese enclosures. Or maybe one of those self immolating SanDisk enclosures. Hell, maybe he’s on a desktop and he slept a raw NVMe on his motherboard without a heatsink

    There are times when you want a brilliant college student on your team, But you need seasoned professionals to help them through the things they’ve never seen before and never done before.

    • Deathray5@lemmynsfw.com
      link
      fedilink
      arrow-up
      8
      ·
      14 hours ago

      Somehow I feel over clicking without understanding of the consequences sounds like something a techbro would do

    • exu@feditown.com
      link
      fedilink
      English
      arrow-up
      74
      arrow-down
      1
      ·
      16 hours ago

      Can’t be a relational database, Musk said the government doesn’t use SQL.

    • iAvicenna@lemmy.world
      link
      fedilink
      arrow-up
      8
      ·
      2 hours ago

      yes but also why say 60K when you could have literally said anything? I mean surely the fact that he thinks 60K rows a big number is already explaining alot lol.

      • Melvin_Ferd@lemmy.world
        link
        fedilink
        arrow-up
        5
        ·
        edit-2
        1 hour ago

        It’s bait.

        They probably have an explanation tweet at the ready to make more sense of it. They just want enough 'hurr durr these idiot" comments before they reverse Uno card this with more context.

        • iAvicenna@lemmy.world
          link
          fedilink
          arrow-up
          1
          ·
          14 minutes ago

          Based on all that has been going on, I feel like they don’t really have the capacity to think more than one step ahead. They do sth stupid and then they usually follow up with “lol joke” or “lol you can’t understand”

    • ameancow@lemmy.world
      link
      fedilink
      English
      arrow-up
      7
      arrow-down
      1
      ·
      edit-2
      5 hours ago

      There are times when you want a brilliant college student on your team, But you need seasoned professionals to help them through the things they’ve never seen before and never done before.

      Honestly, any sweet, white-haired old lady who keeps pictures of her dogs and grandkids on her desk who’s been doing data entry for 15 years could do circles around these clowns.

      But she might also have the wisdom and perception to know we’re not supposed to be doing this “work” at all, which is why he recruits naive teenagers and college kids who are still emotionally immature to think that this is going to be their “destiny” or their opportunity to get into the big leagues of business.

      • rumba@lemmy.zip
        link
        fedilink
        English
        arrow-up
        2
        ·
        3 hours ago

        I keep hearing things about these hires he has, I don’t think they’re naive, At least not as such. They seem to be more power hungry trust fund babies.

        But yeah, people with a few years in them would be a moral liability in that line of work.

        • easily3667@lemmus.org
          link
          fedilink
          English
          arrow-up
          2
          ·
          2 hours ago

          Yeah if you read more of these guys tweets they are clearly in politics. One message tried to claim trump loves kids (to be clear: in the abstract sense, not in the he definitely fucked kids on an island with Epstein sense). Then they tried to twist the words to say “why don’t you love kids”. It was clumsy like you’d expect from someone who is practically a teenager, but the core is an attempt to follow the usual right wing playbook.