Linux, Ubuntu

Ubuntu 14.04’s constant memory struggles

Looking at htop I notice that most of my memory is sucked up in graphics-intensive desktop applications (surprise). Here are the big memory killers I’m going to address:

  1. Browser* – 1.2GiB
  2. compiz – 225MiB
  3. X – 175MiB
  4. spotify – 113MiB

*I’ve recently switched to Opera Developer for performance reasons.

4 gibibytes of RAM depleted within an hour of using Ubuntu

The window manager & graphics server take up 400 MiB of RAM combined just by themselves. Reducing the use of windowed applications kills two birds with one stone: less load on the window and composition manager and lower # of SO’s in memory interfacing with X.

The alternative command line work flow

  • tmux – Rather than opening multiple terminal windows. I use Guake as an omni-present terminal.
  • gcalcli – Instead of leaving the Google Calendar tab open. ccal and pal are good alternatives.
  • despotify – Replaces the resource-hungry Linux Spotify client.
  • vim/emacs – RedNotebook is a great lightweight desktop journaling application, but I still find it preferable to maintain a personal journal using vim.
  • mutt with offlineimap – Mutt and offlineimap for email make a great team. Backing up and archiving all your email using this configuration can be automated with one or two line scripts. Side note: Linus himself uses pine for email.
  • bitlbee & weechat – Bitlbee acts as an IRC server, fully managing all of your chat accounts including Google talk (hangouts), Facebook and Twitter. weechat is then used to connect to your bitlbee server. I found this combination yields the smallest returns on performance but it is an important part of my work flow.
  • elinks – Lightweight cli browser.
  • sc – Great for doing simple spreadsheets and calculations without having to load LibreOffice.

Takeaway

There was an order of magnitude gains in memory usage here. I can run all of these at once and the system’s overall response is still much snappier. If you’re comfortable using the command line I will always suggest test-driving CLI alternatives for your favorite applications.

Standard
Linux, Ubuntu

Recovering ecryptfs-encrypted home files from a LUKS encrypted drive in Ubuntu

I did this in Ubuntu 14.04.

This is a bit of a niche situation and it took a really long time so this guide is incomplete. If you run into the situation and need extra help just comment below and I’ll try to give some guidance.

Restoring with rsync

Restoring with rsync

Recently I ran into problems trying to update to the 14.10 release. I lost my video drivers, x11 fallback drivers, and window manager.

Rather than mess with all that I swapped out my laptop’s drive with a fresh one, enabled logical volume management (which the Ubuntu installation disk says is easier to manage) and then installed Ubuntu 14.04 on the new drive. The old one is now a bodged external drive, but it took some finagling to get it to mount.

The first problem I ran into is both of these drives are now boot drives and the logical volume names are both the same. As much as you try, you cannot mount the logical volumes simultaneously without renaming them. Also when you do run sudo lvscan it will show both conflicting logical volume names (ubuntu-vg/root).

I removed my primary boot drive–that’s the important part–then rebooted into a Ubuntu live disc. You should always make a live disk for convenience. Finally I ran lvrename to change the logical volume name of my external. Once I was done with that I rebooted to my primary drive again.

Bingo! Now when I use the disks utility I can mount both logical volumes. You can’t mount it from nautilus, but don’t bother with it. Next is the harder part: You have to mount your home folder that is encrypted with ecryptfs. Look at the post by idallen for a script to mount your private folders at a new mount point. You will have to make modifications to the script. I also manually set the USER variable in the beginning of the script so that I could sudo the script. Make sure you’re pointing to a place on the old drive where your .Private folder is. AND BE CAREFUL WITH THIS. I made the mistake of creating some recursive directory structures in my destination through symbolic links and I could have just as easily deleted the encrypted home folders the same way.

Once the home folder is mounted, you can rsync -r the contents to a new folder. Do not use cp!

That is all. Like I said, if you run into issues let me know in the comments and I’ll try to help.

Standard
Atomic, Productivity

October Atomic update

I’ve worked on Atomic for months and months now. I’m closing in on an MVPHowever. HOWEVER: I’ve coded an entire working TO-DO manager and I cannot stand to use it. Yes, I know other people can so I will continue to polish it and then release it.

There’s a simple reason why though. To-do lists have to stay out of your way by being minimalist, but if they are too minimalist they will hinder you further. Take these assertions:

  • Pen and paper is very minimalist: pen and paper are always available to you.
  • If you want a to-do list, a pen and scratch paper works great!
  • If you want something with archives and deadlines, a notebook or planner system works great!
  • If you need to share a lot, web can be considered, but eh you can also take pictures of your notes and pass them along.

Ok it starts to get yucky:

  • If  you want to keep track of links, store photos and documents, or collaborate with large teams a web task management system is better.
  • If you need to manually sort these tasks before a deadline approaches (priorities shift all the time!) you’re way screwed. Get ready to drag and drop a lot of crap.
  • If you want to type notes inline with your tasks to work out other stuff you need to do, wow, maybe you should use evernote or something? Clicking back and forth between similar tasks to compare notes will just drive you insane.
  • And what happens if one task turns into a bunch of other smaller tasks?
  • What if the one task should actually be a whole project that someone on your team needs to take time for?
  • We didn’t really have time to go through these little details in the planning phase. Hm.
  • Your big tasks really should be split into smaller chunks. Good luck!
  • If you want to collaborate AND feel free using your task list online there’s really nothing you can use.

The whole problem about atomic is that it treats tasks like–well Atomic units. But tasks behave more like quantum particles; they’re discrete reminders that larger things exist, but when you begin to examine them they start to change and take shape.

Here’s a solid example of something that I ended up doing yesterday:

  • I just wanted to install an application on my linux computer. Initial task: install application.
  • Oh, the build is outdated. I’ll have to compile from source. Task now becomes: compile and install application
  • That depends on other libraries. I’ll have to install those too. Task: install dependencies, compile and install application
  • Ok, all dependencies for my distribution are out of date too! Task: compile dependencies from source, install them, compile and install application
  • You think I’m even close to finish? Nope. Task: update core os libraries, fix circular dependencies, [...]
  • Three hours later I’m reinstalling my OS from scratch and cursing.

Very quickly one simple task can turn into hundreds of smaller ones. I would’ve liked to document some parts of this process, but wrestling with a task manager at the same time would’ve been futile.

I’ll let your imagination run with that. I think Atomic has the potential to fill a much bigger gap while still synchronizing with online services such as Producteev.

Standard
Blog

New format coming soon

It takes too much time to keep up with long form post format I’ve maintained over the years. It’s also really boring.

BORING

Ok. Things I’m working on:

  1. New site (maybe)
  2. New posts (maybe)
  3. Atomic
  4. Probably 100 other things

Going to update in more detail with another post here shortly.

Standard
Linux, Ubuntu

Ubuntu 14.04 LTS (trusty) peculiarities and how to fix them

A small guide to big frustrations.

Shutter (for screenshots) goes slow

This is a problem with your video driver. Run “Additional Drivers” from the unity dashboard and snag the non-generic video drivers. You’ll notice an overall speed boost for everything else too.

Additional Drivers

No swap file with LVM / Locking up in Ubuntu 14.04 LTS

This has to be one of the wackiest things with 14.04 LTS. Check htop or free. Do you even have any swap? If you chose to encrypt your home folder then probably not.  This guide explains how to set up a swap file. Give yourself a few GB (I have 8.) Otherwise if you run out of free memory processes will begin to halt. If you lock up entirely, remember REISUB. Important to note: swap files in the root file system are going to be unencrypted unless you are using full volume encryption. This gives you faster swap speed but leaves the swap vulnerable if someone gets physical access to the drive.

Ah, that's better.

Ah, that’s better.

Takes a long time to mount the swap partition (Press S to Skip…)

If you see this message every time you boot:

The disk drive for ubuntu–vg-swap is not ready yet or not present.

Continue to wait, or Press S to skip mounting or M for manual recovery

This is the same issue as above. Set up a swap file, and also comment 2 lines out of /etc/fstab:

vim /etc/fstab

vim /etc/fstab

Make sure you edit the 2 that start with /dev/mapper at the bottom, but leave the top ubuntu–vg-root volume alone! Unfortunately I’m guessing this is an issue with the guided Ubuntu setup for 14.04. I know the volumes are being created for swap and root but they never mount!

Visual Stutter, lag, slowness with full volume encryption

Unfortunately I resorted to completely nuking my installation and reinstalling without full volume encryption. It would be nice to know if there’s a solution to this aside from getting an SSD.

My personal experience here is that running without full encryption is so much faster that I’m not concerned with it.

 

Update 10/7/2014:

Print screen doesn’t work

I used Compiz Config Settings Manager (CCSM) to bind the printscreen key to Shutter. Unfortunately this introduces a small delay.

Standard
Linux, Ubuntu

Replacing sendmail with postfix on Ubuntu Server 12.04

Sendmail runs as a daemon out of the box on Ubuntu so just installing postfix won’t change postfix to the default MTA. But if you simply uninstall the sendmail packages you will run into the issue of no longer being able to use the `mail` program. Even though you may not use it yourself it’s still required by some scripts to function correctly.

Here’s what I did to get everything working correctly on a headless install:


sudo apt-get install postfix
sudo apt-get remove sendmail-base sendmail-cf
sudo apt-get install bsd-mailx

After that, configure postfix to your liking, restart the postfix service, and finally adjust your firewall settings.

Standard
Linux, Ubuntu

Ubuntu – Getting Password Prompts for Gmail after your Remove California (Calendar)

This is a very specific issue but hopefully it clears some aggravation.

Your calendars for California are still stored in:

~/.config/evolution

If you have evolution installed the calendars will sync. However if you’d prefer to get rid of them entirely you should comb through it and delete the offending calendars. (Beware: If you have local calendars you want to keep, don’t delete the entire folder.)

Here are some other common culprits that cause similar issues:

~/.config/goa-1.0
~/.config/desktop-couch
Standard
Linux

The Heartbeat Bug in OpenSSL (It doesn’t affect SSH)

Thank goodness (see the post by nix); it’d be quite a hassle to reissue keys for git and SSH:

For a change, SSH is unaffected, since this is an SSL/TLS-layer bug, not a core crypto code bug.

Fortunately I only host a handful of services that use the affected versions of OpenSSL. It is recommended that if you are doing anything mission-critical you shut down these services and firewall them immediately. Then reissue new certs for every service(s) after you patch everything up.

Edit: I’m seeing a flood of updates coming in for Ubuntu today (2014/04/08). Basically anything linked with with libopenssl will eventually be updated.

More information [heartbleed.com]

What is heartbleed? [serverfault.com]

Standard
Project Status

Project Updates

Software Projects

  • I halted production on Atomic (cross-platform Campfire Client) as a result of 37signal’s change in product direction.
  • Since I already had a production-ready logo for Atomic, I am going to use the Atomic branding on Eve Task Manager.
  • Eve Task Manager (now Atomic) is coming together nicely, but I do not have time yet to do list announcements. It takes a lot of time to get those emails together, and the demo isn’t ready for an official announcement.

Engineering Projects

  • The production QJ jet engines are ready for prototyping, but the cost of additive manufacturing for the scale tests using Alloy 713C is expensive. I may have to use cheaper materials in the first runs.
Standard
Blog, Productivity

I considered DokuWiki these past few weeks. However after transcribing each of my notes to DW, it was apparent how slow this would be to work with on a regular basis. The biggest pain point was not being able to easily clip articles anymore. Also, hosting a local webserver for it takes up a lot of memory and battery life

If you’re concerned with security, organization, or platform-specific issues on Evernote and considering switching to a wiki, you may want to reconsider it; using a personal web-based Wiki for anything feels disruptive.

I now have all my notes back into Evernote and used a new Stack/Notebook/Note structure to handle Evernote’s arbitrary hierarchical limitations. I keep all of my interests, dev, platform, and projects neatly in their own stacks. Nearly every child notebook has an index, general reference and TODO list. It’s a workflow that I developed in the process of moving everything back and forth.

Ultimately, refactoring all my notes helped me prune old data and refine my ideas and personal projects.

Switching Back to Evernote

Aside