Firstly sorry if this has come up before, I searched both google and the forums but couldn’t find a satisfactory answer.
I’m planning on installing Linux (not picked which yet) on a portable device. I am going to be using it to run python simulations when I don’t have internet access (can’t access the central uni server) some of which are fairly intensive.
So my question is basically a hardware one, is there any advantage in performance in installing on a portable hard drive over a standard USB stick?
I assume the USB port speed itself is a problem but not sure if with a USB3 there is any advantage to a hard drive.
Also as a follow up question, if a hard drive will provide an improvement. I currently have a fairly old USB2 300GB one. Presumably I need to get a USB3 one (or just a cable?) will the cheapest USB3 hard drive be port limited or is it worth spending money on a high speed decent one.
There is certainly a MAJOR difference in speed between a proper install to a USB(2) HDD and a “LiveUSB (with persistence)” … the LiveUSB is slower.
But I couldn’t say for sure how a “proper” installation to a USB stick would compare to a USB HDD.
My guess would be it would be quicker from the HDD, though both would probably exceed the capabilities of the universal serial bus … so there may be little difference.
Ok interesting, I had been planning on going for a live enviroment based on this advice:
[b]Make your choice[/b]
There are two distinct ways of running Linux from a flash drive, but we'll only be covering one of them here – and for good reason, as you'll see.
The two ways are: using your flash drive as a Live CD, and using the drive as a full Linux install. Perhaps surprisingly, the first option is the better one for several reasons:
Live CDs scan the hardware at boot time, and so are likely to be compatible with the most machines.
Live CDs must by necessity have a small footprint, which means there's more space for your files – or you can just buy a smaller, cheaper drive.
Live CDs run as much as they can in RAM, which makes for better performance.
Live CDs don't use swap.
That last point is actually the most important one of all: Linux makes extensive use of hard drives to save temporary data, and if you treat your flash drive as a real hard drive then Linux will use it for swap, too.
The problem is that flash drives, like all flash memory devices, have a limited number of times they can be written to before they fail. Most drives are able to withstand 100,000 writes to every memory sector, but the best can handle up to 1,000,000 and often automatically balance writes to ensure that no one sector gets excessively worn out while others are sitting untouched.
OK, as long as you don’t have a lack of RAM, they are wrong … Linux will rarely page out to the swap partition … indeed you can tell it not to at all.
they are however correct that a “proper” installation to a USB stick IF it were to make use of the swap partition (due to a lack of RAM) would wear it out quicker (this could also apply to an SSD)
A live environment with persistence WILL be MUCH slower than a “proper” install (it will also use more RAM, and be harder to fix if things went wrong)… so ideally a “proper” install to a USB HDD would be your best bet.
[EDIT]
This:-
Live CDs run as much as they can in RAM, which makes for better performance.
Is also not fully correct … both types of installation will attempt to run the application from RAM if it’s available … but the “proper” installation will have access to swap if needed, and will load the application into RAM faster.
A live environment is also having to load/run EVERYTHING into/from RAM, so will use up your available RAM much quicker … then as it can’t use swap it will crash.
Most mobo’s that can’t boot from USB3 will have a USB2 port too … so you’d have a couple of options, boot from USB2 with say a USB stick then get the bootloader to pass to the USB3 HDD … or just use USB2
I may even be totally wrong about booting USB3 … I just seem to recall seeing that somewhere … all I was saying is check first
I am going to be using it to run python simulations when I don't have internet access
I might be wrong on this one, but unless your simulations access the hdd to read data endlessly then wether it is USB2 or USB3 will become irrelevant.
I would expect that the python simulations would read data into memory first and then the actual processing would be run in memory.
Anyway that is how I would tackle it if I were to design the software. Constant IO is a definitely a No-No regardless of the source.
I’m with Sezo … unless your sims are I/O intensive, there’s no real benefit to USB3 over USB2, except maybe slightly quicker load times.
I’d be very surprised if your motherboard hasn’t got a bootable USB slot, and as far as I know USB3 devices are backward compatible, so get either a USB2 or USB3 external HDD … you should be fine.
What I was saying is that if your motherboard can’t boot USB3 … there’s no NEED to get a USB3 HDD if they cost more … but AFAIK it won’t hurt either, just plug it into one of the bootable USB2 ports.
Thanks for your help so far, hopefully one last question.
Looking at instructions for full installs on USB drives a lot recommend physically disconnecting your internal hard drive to remove any risk of accidentally wiping it. I’ve working off a laptop and obviously this isn’t an option. Is this a major concern? What other steps can I take to make sure it doesn’t happen?
OK, you can install with the internal drive attached, you just have to be VERY careful where you tell it to install Linux and the bootloader … as others have suggested it’s MUCH safer to disconnect the internal drive, and this should still be possible with a laptop.
Another (safe) option would be to connect the USB HDD to a desktop PC (that’s had its internal drive disconnected) … install Linux … then attach the USB HDD to the laptop.
(then put the desktop back together)
Unlike Windows which would likely crash if you install on one machine then transfer the drive to another, the drivers in Linux should automagically adjust to suit the PC the USB HDD is attached to … because most drivers are contained in the kernel or as loadable kernel modules, Linux tends to be good at that
(that’s also why LiveCD’s work … the drivers are mostly self contained in Linux)
Ohh ok I had read somewhere that installing it in this way configured it to the system you used and it would not be portable without a lot of faff.
You should bear in mind that by installing a Linux OS on a USB fashion it will not be portable. What this means is that the OS will set itself to use the computer it is first booted from and configure itself as such.
If you take the USB stick after a full install and boot, then bring it to another computer with different hardware and boot from that, chances are high that the internal settings of the OS on that stick will get all messed up because it’s "expecting" a different the computer it was first booted from. Sure, you can simply reconfigure everything and get it working again relatively quickly for anything messed up, but that’s a bit of a hassle.
For each computer you want to boot a full-install-on-USB-stick distro from, it’s recommended you get a separate USB stick dedicated to each computer. The sticks are cheap enough anyway so it’s not a big deal.You should bear in mind that by installing a Linux OS on a USB fashion it will not be portable. What this means is that the OS will set itself to use the computer it is first booted from and configure itself as such.
If you take the USB stick after a full install and boot, then bring it to another computer with different hardware and boot from that, chances are high that the internal settings of the OS on that stick will get all messed up because it’s "expecting" a different the computer it was first booted from. Sure, you can simply reconfigure everything and get it working again relatively quickly for anything messed up, but that’s a bit of a hassle.
For each computer you want to boot a full-install-on-USB-stick distro from, it’s recommended you get a separate USB stick dedicated to each computer. The sticks are cheap enough anyway so it’s not a big deal.
Unfortunately I’ve got a Vaio and its all very screwed together, I can’t even get the battery out without a screwdriver, so I’m not that keen on that route. And I don’t have access to any desktops which I would be allowed to take apart either, I do however have access to a large number of Uni desktops which I doubt I would get in trouble with if something went wrong, so that might be an option.
If I were to do it on my system with the HD in is it just a case of needing to get the right settings? Or is there any chance that even if I select everything correctly I could still junk it?
Whoever wrote that doesn’t seem to have a grasp of how and where from Linux loads its drivers … and is expecting it will behave like Windows.
There ARE some proprietary drivers (that you would have to install after installation anyway) that might make it non-portable … but directly after an installation (before installing any proprietary drivers) it should be FULLY portable.
EVERY time Linux boots it interrogates the PC hardware to discover which hardware is present, and only loads drivers for the hardware it discovers … otherwise how would a LiveCD work on 1000’s of different hardware configurations ?
Linux is VERY good at this, and is highly portable as long as (or before) proprietary drivers are installed … and in most cases, even then
Ignore the quote you just posted … try it and see … you’re in for a pleasant surprise
If I were to do it on my system with the HD in is it just a case of needing to get the right settings? Or is there any chance that even if I select everything correctly I could still junk it?
If you get all the settings right, it’s highly unlikely you’d “junk” the current OS … BUT … there wil be multiple ways you could get te settings wrong.
First there’s the possibility of installing to the wrong drive … the damage that would cause is kinda self explanatory … no more Windows.
there’s also the possibility of installing Linux to the correct drive, but overwriting the Windows bootloader on the internal drive … leaving Windows unbootable without the USB HDD connected
There’s the possibility of not installing the Linux bootloader (GRUB) at all, which will leave Linux unbootable.
The last 2 scenario’s are fixable, but it would obviously be better not to be in that position in the first place
Ok I’m pretty close to giving this a go, just 2 more things, most installation guides i’ve found seem to be geared towards putting on ‘live’ versions and im finding it pretty tough sometimes to distinguish which version (live/normal) the guide is installing. For instance can I follow these steps to get a non live version:
Secondly, this is a big hard drive and I definitely dont need all the space for Linux. Is it possible to partition it so that I can a) keep files on part of it normally and b) share files from linux to windows via a part of the hard drive?
NO, those instructions are for creating a LiveUSB, so after creation when you boot to the stick it will be a “live” environment you’ve booted into.
BUT, you could then use that LiveUSB stick to do a “proper” install … ie. use it as the installation media (same as a LiveCD).
Secondly, this is a big hard drive and I definitely dont need all the space for Linux. Is it possible to partition it so that I can a) keep files on part of it normally
It most certainly IS possible to partition it so it has say an NTFS partition that Windows could use … though the installation may be more complex … I’ll come back to this in a bit.
and b) share files from linux to windows via a part of the hard drive?
OK, by default most Linux distro’s are going to use an EXT4 file system … Windows CANNOT read or write to this file system, in fact it won’t list it as even being present … but Linux has no problem reading and writing to Fat/Fat32/NTFS partitions … so the Windows drive/partitions will be visible from Linux, but the Linux partitions won’t be visible from Windows (at east without third party software)
So…
As the external drive is large, and Linux doesn’t really need much room … I agree, partitioning to have a separate NTFS partition would be a good idea, at least then it could still be used to tansfer stuff between Windows PC’s without having to boot Linux.
Make sense so far ?
Now it’s just a matter of the best way to achieve this … you have 3 options … 2 of which are slightly easier -
the way that may confuse … during the Linux installation, set up the partitions how you want, creating say a
38gb / (formatted as EXT4)
and a 2gb swp (swap)
then create an NTFS partition using the rest of the drive.
Install Linux using ALL of the drive, then once installed and working … use a LiveCD/LiveUSB to resize the partitions with Gparted.
Use Windows disk manager to create an NTFS partition using all but 40GB of the drive, leaving 40GB unpartitioned … then during the Linux install, select “use the free space” … linux will then create it’s own partitions on just the unpartitioned space
I’d go for option 3
And if any of that didn’t make sense … feel free to ask
Ok thanks for all your help so far, I think im just gonna man up and take my internal hard drive out, so I think im just about set to go. I may be back with questions when it all goes wrong (hopefully not)
Ok so I’m half way there. I’ve partitioned the hard drive and Loaded the live version onto the windows bit of the hard drive. Ive also changed the bios order so that boot from external device is first. This all works fine, when the internal hard drive is in and the usb is plugged in it boots straight to ubantu.
However when I remove the internal Hd To do the full install, all I get Is ‘operating system not found’.
