You must log in or register to comment.
- Boot to usb
- Mount your root filesystem
arch-chrootyour mounted root filesystem- mount
/boot mkinitcpio -p linux
Steps 1,2 and 3 are the entry way to solve all “unbootable Arch” problems by the way, presuming you know what needs to be changed to fix it of course.
For a while, I had to do this after every kernel update
Turns out, i accidentally had two
/bootfolders. One was is own partition, and the other was on the rootfs partition. When Arch booted, the separate partition was mounted over the rootfs/bootdir, “shadowing” itExcept, UEFI / GRUB was still pointing to the rootfs partition. So when pacman installed a kernel update, it wasn’t able to update the kernel that UEFI was booting, but it was able to update the kernel modules
Kernel no likey when kernel modules are newer than the kernel itself
I’d gladly take an Arch wiki article
shutdown a computer when you shouldn’t computer breaks
how could a computer do this
To be fair a lot of Linux distros and other operating systems try to be careful on how they do things. Arch is the odd man out.
How exactly?
Pacman is very fast at the cost of safety checks and verification
Power outages do happen, and I’m pretty sure 90% of the people on this community are not using an UPS.
Given enough users and enough time, it’s inevitable that a power outage will happen to some people at an inopportune moment, like while updating an important package like the kernel.
Blaming the user for this is not fair, it’s just dumb bad luck.
That said, OP could have done a bit more to fix the issue instead of being an angry man yelling at the cloud. When you’re using Arch, the expectation is that you are able to fix relatively simple problems like this, or that you’re at least willing to learn it. If you find yourself getting angry when Arch doesn’t hold your hand, you probably shouldn’t have chosen Arch.
Why would Arch do this?
Pacman is very fast. The tradeoff is that it isn’t as “thoughtful” when it runs. It doesn’t have the same protections as apt or dnf. I especially like dnf as you can undo any operation.
The power is out and my laptop has less than 10% battery left?
It’s pacman -Syu time.
What about a desktop PC?
Exactly my thoughts as well.
Why update on that little battery life left… the power will return sooner or later, going without updates even for a week or two is no real problem. Hell, I update like once every 3 weeks to a month, it’s not that big of a deal.
I update always after a session, meaning about once a month 😂I don’t really need my PC, lol
Wait if the power is out, how do they have Internet to load new packages? Something doesn’t make sense here
My router has a 5G backup connection and a battery. Light could be out and I’d still have internet. So, yeah, it’s possible :P
But if the power goes out isn’t the stuff from your ISP/mobile carrier out too?
Nah, cell towers often have some kind of backup power good for a couple hours or more, at least in the city where I am. I think I once had an outage last 3 days when a tornado wrecked the local transformer station, and still had cell service the entire time.
It first downloads all packages from net, then it proceed totally offline starting by verifying downloaded files, signatures, extracting new packages and finally rebuilding initramfs.
Because arch is replacing the kernel and inittamfs in-place there is a chance that it will not boot if interrupted.
This issue was long resolved on other distro.
One way to mitigate it is by having multiple kernels (like LTS or hardened) that you can always pick in grub if the main one fail.
Yeah but how can you download without internet
(No internet =} no download = no failure
You can even host repo mirror locally, that will still work without internet ;)
How to have internet without power?
- Mobile hotspot
- Router and AP using UPS
This issue was solved on Slackware in 1993.
It installs a “huge” kernel that contains all drivers to run on almost any hardware by default, alongside the “generic” kernel with only the modules you need. If the generic kernel fails to boot, you always have the backup, which is known to work, cause it’s the kernel you first boot into after installation.I’m not familiar with slackware but why is specific kernel called generic, while generic one is not called generic? I’m puzzled
I have no idea either.
Edit: Did some reading. “Linux-generic” is just the name of the linux kernel that is used in most computers (as opposed to Linux-realtime, which is the only other Linux kernel that’s still relevant).
Cable internet tends to stay online even if your power is out. You’d need a battery backup for your modem/router, but it is possible to stay online. Houses can be clever like that, almost all of your utilities will partially work, even when service is interrupted.
That depends on the ISP having backup batteries for their equipment. It will usually only last a couple of hours. 5G will usually stay up for a few days. For longer outages, you will need satellite internet and lots of fuel for your generator.
Average Linux solution.
“Got an emergency? It’s so EZ. Just open up the terminal and copy/paste [long string of unreadable text]. Btw fuck windows.”
Yea as opposed to the windows method of “just open regedit and navigate 8 folders below HKEY_CURRENT_USER to change some ambiguous system variable in hex” lmao
I’ll take editing a text file in /etc/ for my configuration any day
Cellular data
I don’t think I’ve had a pacman update take longer than 10 minutes before. Sounds like OP was updating all their AUR packages too.
Still absolutely a terrible thing to do on 10% battery life. I bet there’s an AUR package for “check battery level before update” out there somewhere though.
OPs meme is "use distro whose model is ‘give users enough rope to hang themselves’ " and complaining he’s at the gallows
I imagine being at the gallows sucks even if you know why you are standing there
When I used Arch I updated once and it removed the running kernel and its modules. So when I plugged in a webcam it didn’t work, since the module was gone.
Not a catastrophe, but it was an off-putting user experience coming from Debian. Arch felt more like a desktop OS, Debian feels more like a server OS to me (updates generally warn/confirm when you need to restart services or the machine).
To each their own! Having more up to date stuff was a nice perk of running Arch, certainly.
Debian and Fedora are solid on the desktop
Oh I love Debian on the desktop! More a comment on the feeling of the OS being very concerned about downtime and stability, with minimal “surprises.” Not a bad thing at all!
I have LTS and zen kernels installed in addition to the default Arch one, that should prevent this yes?
I’m not sure - I had zen and the default kernel and both were inaccessible
This got me looking to see if there is any way to have a fallback as I have had something similar happen to me.
The general advice is to have a liveboot USB around. I even saw that you can have GRUB simply boot from an .iso file on the internal drives, which eliminates the need to keep a USB stick around.
I haven’t followed the steps yet but I’ll give this a shot because it intrigues me.
https://www.linuxbabe.com/desktop-linux/boot-from-iso-files-using-grub2-boot-loader
Or you could just install NixOS for update rollbacks (or use zfs/btrfs and set an alias to take snapshots whenever you update)
I always have a separate huge kernel on hand that boots without an initrd.
This is why you keep a backup kernel
If you’re planning for this type of failure, what you probably want instead is Aurora from the Universal Blue project. Since it’s fedora silverblue underneath, your OS either updates all at once or doesn’t.
Out of curiosity: Which operating system(s) can you shutdown while the kernel is being overwritten? I wouldn’t imagine that as a limitation of Arch Linux specifically.
I think fedora would survive this abuse. It doesn’t replace when you install kernels, but instead adds it.
Also Fedora ships 3 kernels by default. If one breaks, maybe the others will keep working.
With Manjaro you choose how much kernels you want.
Arch let’s you install kernels till /boot is full…
Yes. I have it set up this way. I forgot it wasn’t the default. For the amount of headache it would solve, I wonder if the Arch team has a specific reason for not keeping a number of previous kernels by default.
Ubuntu (and probably Debian too) will keep an old kernel in your grub list so you can boot off that one if needed.
Mint definitely keeps a couple of previous kernels around, so that might be a Debian and Ubuntu thing too.
That said, there’s always going to be a critical point of failure that a power loss could cause things to break, no matter your OS or distro.
Writing the bootloader or updating a partition table for example.
Anything running on a copy-on-write filesystem can trivially rollback changes using a rescue partition.
I also expect most immutable distros would be able to be especially good at tanking this.
I assume NixOs would just let you load a previous working configuration if the current one got corrupted (though in this case it probably could simply rebuild the current one).
Windows
Goes back to a previous restore point
I haven’t used Windows since Win7 - Is it possible nowadays to immediately cancel a kernel-level upgrade (say, Win7 to Win8) and have it gracefully stop and then boot into the pre-upgrade environment? If so, then Windows has come a long way. We use to be careful breathing-too-loudly around Windows computers during the upgrade process. Microsoft must be getting better.
Arch Linux with 2 kernels ;)
I think I didn’t make it clear enough: My laptop was on the power during the update process, when the power randomly cut out - for the first time in about 6 years, it doesn’t happen often. Of course you can interpret it as user error - but I think it’s reasonable to update my system when plugged into, normally reliable power. The laptop battery is pretty much dead, so it would’ve shut itself down automatically anyway.
sure, but what os wouldn’t break if you did this?
Plus in Linux you can actually fix this with a live USB, while on Windows you can run startup repair and hope for the best.
In Windows you can also fix this with a live Windows USB, manually.
If it was on something like BTRFS it’d probably be fine, though I imagine there’s still a small window where the FS could flush while the file is being written.
renameat2has the EXCHANGE flag to atomically switch 2 files, so if arch maintainers want to fix it they could do- Write to temporary file
- Fsync temporary file
- Renameat2 EXCHANGE temporary and target
- Fsync directory (optional, since a background flush would still be atomic, just might take some time)
it was btrfs.
Just having btrfs is not enough, you need to have automatic snapshots (or do them manually) before doing updates and configured grub to allow you to rollback.
Personally, I’m to lazy to configure stuff like that, I rather just pick my Vetroy USB from backpack, boot into live image and just fix it (while learning something/new interesting) than spend time preventing something that might never happen to me :)
renameat2
I read this as “rena meat 2” and was very confused
Windows doesn’t in my experience, it’s surprisingly robust.
But also I thought Linux distros normally keep the old Kernel around after an update so stuff like this doesn’t cause a boot failure?
Yeah windows “cumulative update” upgrades for the past couple of years basically duplicate the whole system directory and apply the update to that leaving the existing one to roll back to if anything fails
But also I thought Linux distros normally keep the old Kernel around after an update so stuff like this doesn’t cause a boot failure?
Arch has no concept of “previous package”, so it doesn’t do this.
You could install
linux-lts(or one of the other alternative kernels) side by side with thelinuxpackage, so you always have a bootable fallback, but like most things on Arch it’s not enforced.That’s pretty wild, I guess arch is not meant to hold your hand at all so it makes sense.
Windows updates (and Windows Installer) are transactional. If the update or installation fails, it knows exactly how to revert back to the previous state.
Windows Installer supports this across multiple packages too - for example, a game might need some version of DirectX libraries which needs some version of the Visual C++ runtime (probably showing my age because I doubt games come bundled with DirectX any more). If one of the packages fails to install, it can handle rolling everything back. Linux can sometimes leave your system in a broken state when this happens, requiring you to manually resolve the issue - for example, on a Debian-based system if the
postinstscript for a package fails.
Any immutable distro, Debian, Ubuntu, all their derivatives, Fedora, all its derivatives, OpenSUSE, Slackware, …
Basically, 95+% of installed Linux systems would retain the old or a backup kernel during an upgrade.Any immutable distro, Debian, Ubuntu, all their derivatives
Debian and Ubuntu are not immutable distributions by default, unless I am mistaken.
Any immutable distro and Debian and Ubuntu and all their derivatives
They weren’t saying Debian and Ubuntu are immutable - they were saying “any immutable distro”, “Debian”, and “Ubuntu” as three separate items in a list.
Ohh, I’m dumb
deleted by creator
good answer to a bad and uninformed question, thanks.
Just about any Linux I’ve ever used keeps the previous kernel version and initrd around. And nowadays snapper makes a new snapshot before and after every package installation or update.
So, I’d think there are a lot.
So what I’m hearing is install Linux-LTS and pacsnap
I still don’t get the problem. Are you complaining you have to chroot into your system and finish the update because your power got interrupted? Is a 5 min detour into a live system making you unconfortable? This is how you would fix it in any distro except the image based ones and the arch wiki will guide you excellently how to do it. Good luck!
I don’t really get why you couldn’t pick one of your other installed kernels and boot that, but you seem pretty intent on blaming arch and I don’t feel like trying to troubleshoot it, so that’s that I guess.
How dead are we talking here? Even on an older laptop a kernel update doesn’t take that long. Should have just kept it going, hoping for the best.
I am using an old laptop that gives me 3 minutes to run from one power plug to the other before just going out.
I mean any which way you try to frame this, saying that you won’t use Arch anymore because you didn’t take the precautions necessary based on your situation is gonna take some heat here.
What precaution would you expect OP to would’ve done though? A fallback kernel would be my guess - that’s something many casual oriented distro do out of the box basically. . I read your post as “you’re right, don’t use arch” - something btw which I tend to agree with although I wouldn’t say that’s because of the precautions.
I use arch because there’s no black box magic. For an end user who expects or wants that… Yes, arch might not be the right choice.
I don’t think lack of precaution was the issue here given that it was an unexpected power failure, but it is a fairly easy fix with a chroot.
Oh agreed! That’s why I’m with OP actually that arch might not be the right distro to go for.
The person I replied to basically said “that’s what you deserve for not doing it properly” if I understood it correctly - that’s what I’m confused about as well.
Yeah it seems half the commenters missed OP’s clarifying comment and just think he started a kernel update with 2% battery life.
Hehe true. And even that happened to me after a couple of tired “Syu enter”. But then again I learned something new with nearly every repair!
If you know your battery is shot and you don’t have a way to save your install if the power goes out, then you just don’t update. There are plenty of ways to protect against this that have already been mentioned (battery backup, backup kernel, etc). OP was just playing with fire.
That’s kind of overzealous. I would expect most desktop users to run kernel updates without being plugged into a UPS, this is functionally identical. It’s not like it’s an unrecoverable error, but yeah if you’re updating a critical system you should have redundancies in place.
How would you set up a fallback kernel in Arch?
I have set up an lts kernel in addition to the zen I use by default. See:
https://wiki.archlinux.org/title/Kernel
Disclaimer: this only works when something with image creation goes wrong with an update. Which didn’t happen to me ever - unless I did a mistake or tested some kernel stuff. I only had bootloader errors when I screwed up pacman though. The fallback kernel in that case is on a USB stick…
shutdown
‘shut down’, here. ‘Shutdown’ is a noun missing a hyphen.
shutdown is a command, so op is technically right.
Commas go inside quotes sir.
They don’t have to. In fact, it makes more sense for them to be out the quote, unless they’re part of the quote. Many writers use commas outside of the enclosement of the quotation marks. I do
deleted by creator
MFW I say more than “L2S” and get downvoted by projectors.
Oh no, a non-english speaker did something that could barely be called a spelling mistake! I’m so sorry for writing “shutdown” instead of “shut down”. 🙏
That’s why UPS boxes exist … Or Timeshift if you don’t have the cash
Image says “laptop”. op could have just charge the battery a but before running the update.
In a comment he clarified that the laptop was plugged in and there was a power failure. Regardless, chrooting in should be a fairly straightforward fix.
If the laptop is upgrading the kernel so slowly it shuts down in the middle of a power outage it’s probably old as hell and has barely any battery to charge.
“Arch is stable”
So I’m trying to understand if you think that shutting down an update during regenerating the initramfs indicates that Arch isn’t stable? Because that’s a FAFO move and would crater any non-atomic update distro.
It doesn’t ruin Debian or Fedora as they do the bootloader last
If it is interrupted it just boots the old kernel
It is! My Desktop hardly ever topples over!
Stable does not mean it’s for everybody. My installation runs since now 10 years.
(The only other distribution this failsafe I know of is Debian)
When talking about Linux, “stable” usually means “doesn’t have major changes often”, or in other words, “doesn’t have lots of updates that break stuff”. That’s why “Debian stable” is called that. Arch is not that.
Ive been here. U can use a bootable usb to boot. Then use switch root to change to ur actual filesystem (I’m glossing over a lot of complications here ask chatgpt) and update from here or just copy over the kernal.
ask chatgpt
You mean read the Arch wiki?
Read the response from the thing that read the Arch wiki
I mean ask the self hosted dolphin finetuned mistral 8x22b but chatgpt is easyer to say.
Why the fuck are you asking an LLM to help you fix your Linux install – especially a tiny one that gets facts wrong as often as Dolphin does – when archwiki is right there?
Dolphin is a dataset to fine tune models in an attempt to revert much of the alignment. The base model is mixtral 8x22b. Its way faster as it gives me the exact commands I need.
I’m not even an Arch user (I use Debian and Fedora) but the Arch wiki is amazing.
I was installing Nobara 40 and discovered that the live session is allowed to suspend the PC during the install process. The system ended up having problems with some basic functions…
