![[info]](http://l-stat.livejournal.com/img/userinfo.gif){kind=small}
How do I know if I have the H1N1 flu?
The symptoms of this influenza virus are similar to just about every 'flu' bug out there. Common effects are a cough, sore throat, headache, fever and chills, severe fatigue with body aches. Some people are over the fever and chills phase in about 2 to 3 days while others suffer for more than a week. Complications can occur days into the illness when lower chest congestion progresses to pneumonia. This is a secondary bacterial infection that comes about because the immune system is so taxed fighting the flu it cannot fend off the bacterial pneumonia. This disease is so prevalent that hospitals and the health department have stopped testing specifically for H1N1 2009. It is always coming back positive, so if you have the above symptoms you are presumed to have H1N1 influenza.
(Yeah, whoever wrote that article needs to work on their language skills.)
Cough, sore throat, fever and chills, severe fatigue with body aches. All there, at one point or another in the last two weeks. Pneumonia last week.
Doctor's appointment tomorrow, because I hate the idea of missing a third week of work.
I've got a 3TB RAID5 volume (three 1.5TB disks) that reads between 150-200MB/s, but only writes at 25-50MB/s.
I would like to have full backup capacity of all 3TB of data, but the question becomes "how"?
If we assume that the reason for the slow write speed to the software RAID 5 array stems from parity calculation, then it stands to reason that a RAID 0 array wouldn't suffer the same speed limitation. Additionally, a RAID 0 array of two 1.5TB disks would hit a 3TB volume size, as opposed to requiring a third disk as in RAID 5.
I'm considering having a second, weaker box run software RAID0, and do a nightly rsync from primary box to the backup box. A dedicated 1Gb/s link would facilitate the copy.
If a drive in the RAID0 array fails, I replace it, rebuild and re-run the backup. If a drive in the RAID5 array fails, I replace it and rebuild. If the rebuild kills a drive and the RAID5 fails, I've got a backup. Meanwhile, I've got an isolated power supply, reducing the number of single points of failure. I'm using fewer drives in the backup machine, reducing cost. I'm reusing older hardware for the backup machine, reducing cost.
Tricky part is figuring out offsite backups from there, but my data isn't that valuable yet.
By "near-proximity", I mean having it attached to the original IR transmitting device, have an IR sensor, convert that to a 2KHz 1-bit bitstream, send that via BT to a receiver that converts it back to an IR transmission near whatever device needs to receive it.
Two significant problems remain: Bulk and power. For bulk, one could take advantage of paper-ICs or other film integration. (When I was a kid, I saw thick-film ICs dating back to the 70s. They may have been around longer than that.)
For power, I don't know. Probably the best way to go about it is to leech off the existing remote control's battery pack. I can think of a couple ways one might do that without interfering with the remote's internal expectations of its power source.
Of course, you could just build the thing into a lithium battery pack, rechargeable via USB, and tout it as both a range and life extension of the remote. Lithium power density is such that you might be able to pack the lithium, charging circuitry and bt transmitter in the space of a couple AA batteries. Some mechanical finagling and shoe-horning might be necessary to fit different battery compartment configurations.
Of course, an order like that comes to about $70, and I don't have that laying around in cash. PayPal was inconvenient for technical reasons, so we came up with a convenient solution...I just bought an equal amount of stuff from his Amazon wishlist, and am having it shipped directly to his registered address.
(I'm going to abbreviate this, including only the steps that worked, and initially omitting some of the excruciatingly long retries)
After installing the three disks into the machine, my first step was using Ubuntu's Palimpset Disk Utility to build the software RAID device. That took about seven hours, unattended.
The next step was copying my old /home filesystem to the new RAID array, using dd. That took about nine hours, unattended.
The next step was expanding the filesystem and tuning some parameters. ext3 isn't aware of the size of the block device it sits on, but it does remember a similar value related to that of the device it was created on. I had to use resize2fs to expand it from having sat on a 1TB volume to occupying a 3TB volume.
I looked at tune2fs and enabled a few options, including dir_index (I have a few folders with thousands of files in them), sparse_super (That saved a *lot* of disk space) and uninit_bg (Supposed to speed up fsck). I didn't read the man page clearly, and didn't discover until afterwards that by enabling uninit_bg, I'd given tune2fs the go-ahead to convert my filesystem from ext3 to ext4. Oh well...Seems to be working, and there are a few features of ext4 (such as extents) that I expect will come in handy.
The next step was to reboot and ensure that I could mount the array after rebooting; I didn't want some screw-up on my part to lead to all that time being wasted* by failing a RAID volume. After establishing I could mount it, it came time to modify mdadm.conf. and seeing that the array would come up on bootup. After that, all that was left was modifying /etc/fstab to mount the RAID volume at /home, rebooting, and restoring compressed tarballs and such from my overflow drive.
Filesystem Size Used Avail Use% Mounted on /dev/md0 2.8T 1.2T 1.4T 47% /home
I've gone from having 8GB free on /home to having 1.4TB free. Can't complain.
root@dodo:/home/shortcircuit# dd if=/dev/md0 of=/dev/null 10985044+0 records in 10985043+0 records out 5624342016 bytes (5.6 GB) copied, 27.5768 s, 204 MB/s 18288001+0 records in 18288000+0 records out 9363456000 bytes (9.4 GB) copied, 46.1469 s, 203 MB/s 22992060+0 records in 22992059+0 records out 11771934208 bytes (12 GB) copied, 57.7066 s, 204 MB/s
Getting over 200MB/s in raw streaming read. Can't complain about that, either; I only read at about 70MB/s when pulling from a single (mostly) idle disk that's not part of the array.
Of course, it's not as good as I'd get with a hardware RAID card, but it's a heck of a lot better than I'd get otherwise. My comparative write speed sits down at about 25MB/s when dd'ing from a USB drive to the md0 device. I probably should have tried testing the write speed while reading from /dev/zero before putting the filesystem in place, but the bonnie disk benchmark at least gives some non-destructive results:
Version 1.03c ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
dodo 16G 38025 73 38386 8 25797 5 47096 85 161903 16 353.8 0
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
dodo,16G,38025,73,38386,8,25797,5,47096,85,161903,16,353.8,0,16,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++For ext4 on top of the software RAID5 volume (consisting of three Seagate ST31500541AS), I get 38MB/s sequential output, 161MB/s sequential input, and 353 random seeks per second. Little to no difference between per-character writing and block writing.
Version 1.03c ------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
dodo 16G 50964 96 61461 15 29468 6 49902 87 84502 6 202.1 0
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++For ext3 on top of a single disk (a Seagate ST3500630AS), I get 51M/s sequential per-character write, 61M/s sequential block write, 50M/s sequential character read, 84M/s sequential block read, and 202 random seeks per second.
Long and short of it, a single block disk kicks my software RAID5 volume's butt for sequential write, but the software RAID5 blows away the single disk for sequential reads, and gets a 50% improvement over the single disk's random seek rate.
One thing I find particularly interesting about this is that the three disks in the RAID volume are 5900 RPM spindle speed, while that single disk is 7200 RPM spindle speed. I suppose having three heads is better than one. :)
As I only have 1.5TB of raw disk *not* part of the RAID, I'm going to peek at compressed filesystems for the 1TB disk. Trouble is, it takes 11 hours to copy a 1TB volume *to* the RAID via DD; I don't think my backups can be daily...
In other news, it's interesting watching SMART data on the drives. The "head flying hours" counter is already higher than the "power on hours", 2.2 days vs 1.8. Go figure.
I'm only going to talk about gzip/DEFLATE because that's the only general compression algorithm I've considered in this approach.
It occurred to me there's a way to make a tarball (and, indeed, any DEFLATE stream) seekable. It stems from the reason DEFLATE isn't really seekable in the first place; The actual encoding of the data depends on data that it's already seen, so you can't just peek at any one place in the stream and start decoding there without knowing what came earlier.
In the case of DEFLATE, there's a compressor state that keeps changing as the compressor processes more data, in order to improve the compressor's efficiency. That state represents how the compressor will encode the symbols it sees in the near future, and the symbols that it sees will cause it to change its state to be more efficient for the symbols following.
In the process of decompressing a DEFLATE stream, that same state gets reconstructed, referenced and updated as a decode key as the stream continues. One can't normally jump to the middle of a DEFLATE stream because one needs to have that state in the form that it would be by that point in the stream.
The solution is simple; Save off a copy of the compressor/decompressor state wherever you want a seek point. Keep an index containing the original data stream address, the deflate stream address, and the decompressor state one would have at those two points.
Put the index in a separate file, after the terminator indicator, or multiplex it using some sort of container format; I don't care. Point is, you have the option of resuming a decompression run that you never really started.
Yes, this harms the compression efficiency, in a way. To seek, you need that decompressor state, and that decompressor state will have a nonzero size. No worries; There are all kinds of ways to tune how large the index will be:
- You could set an index size as a percentage of the resulting deflate size. After building N bytes of deflate stream data, you save off an index point of M bytes, where (M/(N+M)) is your target percentage.
- If you know enough about your source data stream, you could be selective about where you put the seek points. For example, if you know that the source stream is a tar file, you can put a seek point at the beginning of each file in the tar archive.
- You don't have to generate the seek index at all until you'd like to start examining the file. Most of the tools I've used that allow be to browse tarballs decompress the entire archive into a temporary file or directory. In such cases, generating the seek index as an initial step saves the disk cost of a temp file or directory, and the seek index can be kept for later reference.
Another interesting side effect of the seek index is that it allows parallel decoding. If the underlying I/O subsystem is up to the task, multiple chunks of the overall DEFLATE stream may be decompressed simultaneously, with each seek index point as a potential starting point.
"Don't you feel alone?"
"Not since I found the Internet."
Steve Martin does fine as the senseless, slapstick-mode Clouseau, and I think only my young age might let me get away with saying he's potentially on par with Peter Sellers in the role; I really think it's the script and the poorly-done special effects that bring down the movie. It does give an interesting perspective of the type of person this Clouseau might be, though: Focused on details, but sees a completely different set from those around him, and has misplaced priorities in other areas. Also has an intellectual understanding of form and social protocol, but a major disconnect between his intellect and actions.
Of course, the classic big band Pink Panther theme is a pleasure to listen to.
