Redesigning the Jet Engine Project

Posted on by

Turbine Blades - Detail

When I first started designing this jet engine project my main goal was to create it using only easily-obtained scrap and raw material parts. I would have to develop a manufacturing process that involved no advanced metalworking to speak of. Handcrafting each part using only basic tools would be laborious. Testing and rotor balancing would also require that I build all the test rigging myself. Any other pieces like oil and fuel pumps would be scavenged from scrap yards or hardware stores. After that, the finished engine would have to work just well enough to handle self-sustaining idle speeds to be considered a success.

At present there is no denying the feasibility of creating your own jet engine from scratch. Soup can jet engines all the way up to full turbo-prop engines can be made from little more than basic scrap metal and cheaply available carbon fiber polymer resin. Usually people have the help of modern lathes and rented CNC machines, but less precise hobbyist projects are all over YouTube. Some of them are very impressive.

Back to my initial goal though: why would I want to use stone-age building tools to build a jet engine? Well, for one, it was attractive from an artistic standpoint. Generally rotors found in turbines have to be precise to a very fine level. This is generally beyond what a person can do by hand and is left to special balancing jigs. Necessity being the mother of invention, I was curious to see if I could develop a technique to balance the rotors and trim off blade metal to make a perfectly balanced rotor without using commercial equipment. Precision hand-craftsmanship was one of the reasons why I had chosen this requirement.

The other reason was slightly more abstract. I wanted to create something from essentially nothing. To explain this a little bit better, someone once relayed the idea to me long ago that “all humans are just chemical reactions happening on a rock in outer space.” While technically true, it simplifies things to the point of nihilism. As the old adage goes, the whole is greater than the sum of its parts. Humans are obviously more interesting than basic organic chemistry, and likewise, turning a once useless pile of scrap metal into a working, moving engine is interesting from a philosophical standpoint.

So there you have it. Two valid points of why I wanted to produce this project from scratch. Lately though, a more practical viewpoint takes precedence in this project. That is, I just want to build the damn thing.

No more messing around here, then. I am working on new designs that won’t require cold forging turbine blades from sheets of aluminum. There will be 3 designs. The first will be your typical turbocharger jet engine [YouTube] made from spare automotive parts. The next two will be designed from the ground up in CAD and the machining will be outsourced to a fabrication company. The parts will be assembled at home and potentially used in an even bigger project. I’m not going to bother writing that one up yet until this one is farther along, though.

Maintaining a Personal or Project Journal in Linux

Posted on by

Linux Journal

Lately I spend most of my time in a Linux terminal. One of Linux’s strong points is that you will always have a way of writing at hand no matter what. Just open up a terminal and type “vim” (or your own editor of choice) and you can type anything, save it and quit. The text you save can be a script, some code, a to-do list, a phone order for Chinese food–whatever you decide.

Late last year I decided to store a day-to-day account of what’s going on in my life and how I feel. It’s given me a lot of insight already and it’s a great way to remember when stuff happened. I originally started this in Google docs, however after a certain length Google docs can become extremely slow. Hell, Google docs is slow all the time. I realized that splitting the files up would be necessary. I wasn’t ready to do that with Google docs because it would make them impossible to search and sort through. I opened up a terminal and made a series of nested folders.

I started with a journal folder in my home directory. Then I added 4-digit year, then 2-digit month. Days were then simple text files stored under each month folder. It looks like this:

~/journal/2012/08/30
~/journal/2013/05/21
…

Note that nothing would really stop you from putting these text files all in a single folder or grouping them all by year, etc. It’s up to you to organize them how you like. I recommend that you keep the date format, though (YYYY-MM-DD) because alphabetical sorting of the files would then be the same as chronological sorting.

The flexibility of this system means that you can create a new entry just by opening up a text editor to the day:

vim ~/journal/2013/04/25

You just write what you need in there and save.

Some Tricks

You can do this for more than just a personal journal. You can use it to chronicle a project you’re working on, keeping a dieting journal, or using it as an hour tracking or work log.

Want to read all of your entries from a particular month?

cat ~/journal/2013/04/* | less

My advice is to use short, succinct sentences, and limit to one sentence per line. Use your words to describe a situation, your feelings, your progress, and your hopes. Don’t limit yourself and be honest because this is for you. Using this brief method will help you describe something quick and easily without turning something into a piece of prose. It will also make it more searchable using grep.

Keep writing quickly for as long as you want, and in the order that it comes to you. You may also group segments of thoughts by inserting an extra new line between paragraphs.

Example

I spent $56 on groceries today
It made me go over budget
Upsetting, but hopefully I have enough of a buffer to make it work

This project at work is carrying on way too long
The reason why is because the project manager missed a detail in the quote
I’m usually the one who gets blamed for it
Not today

While keeping track of details like this is good, a better example would involve a little more depth, introspection, and feeling. Something beyond what you would share on a blog. You will be amazed at how many things you lose track of on a daily basis.

Here’s an added bonus. I wrote a Python script that will compile all your journal entries from every year, month, and day into a single HTML file. It assumes you’re using the same folder and file structure that I do (~/journal/YYYY/MM/DD). Just drop it into your journal directory and run it to create a fully searchable, no-frills HTML file. Load it up in Lynx to search or scan through your entries by typing:

lynx index.html

Also note that it has FTP support in case this is being used to generate project logs. It’s commented out by default, but I use it to keep track of the VTL flight computer software that I’m working on. I even created a simple cron job that compiles and uploads my progress every night.

Here’s the script:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 
#!/usr/bin/python
 
import glob
 
# uncomment for FTP support -- requires ftplib
#from ftplib import FTP
 
output = open("index.html", "w")
 
html_escape_table = {
	"&": "&",
	'"': """,
	"'": "'",
	">": ">",
	"<": "&lt;",
}
 
# Credit: http://stackoverflow.com/questions/2077283/
def htmlEscape(text):
	return "".join(html_escape_table.get(c,c) for c in text)
 
def startBody():
	output.write("<!DOCTYPE HTML>\r\n<html><head><title>My Journal</title></head><body>")
	output.write("<h1>My Journal<h1>")
 
def endBody():
	output.write("</body></html>");
		
def startEntry(date):
	output.write("<h2>" + date + "</h2><!--startentry-->");
 
def endEntry():
	output.write("<!--endentry-->");
 
def insertParagraph(paragraph):
	p = paragraph.strip()
	if p != "" and p != "\n":
		p = htmlEscape(p)
		output.write("<ul><li>")
		output.write(p.replace("\n", "</li><li>"))
		output.write("</li></ul>")
		output.write("<br>")
 
#BEGIN
startBody();
 
#list all journal files and sort them in reverse chrono order (newest first)
#directory structure should be in ./YYYY/MM/DD format like so:
#./2013/04/31
files = glob.glob("./*/*/*")
files.sort(reverse=True)
 
# loop through journal files
for fileitem in files:
	# read the journal file
	f = open(fileitem)
	j = f.read()
	j = j.replace("\r", "")
 
	# split into paragraphs 
	j = j.split("\n\n")
 
	startEntry(fileitem)
	# loop through paragraphs
	for p in j:
		insertParagraph(p)
 
	endEntry()
 
endBody()
 
output.close()
 
# additional feature!
# uncomment the code here to upload this file to an FTP site:
 
#ftpuser="FTPUSER"
#ftppass="FTPPASS"
#ftphost="example.com"
 
#ftp = FTP(ftphost, ftpuser, ftppass)
#ftp.storlines("STOR index.html", open("index.html"))
view raw compilejournal.py This Gist brought to you by GitHub.

Fixing the Producteev Windows Application

Posted on by

Today I’m writing about the Producteev Windows Desktop application for task management. Producteev for Windows allows you to bring the web-based task management software to your Windows PC.

The current release has issues for some users. I will address the issues in this post, as well as their symptoms, causes, and the architectural changes that were required to overcome them.

If you came here looking for an update to Producteev for Windows: Don’t worry, it will be out soon! The remainder of those post is more of a technical paper for those interested.

This post might benefit a few people, especially if you’re having performance trouble with your C# application, seeing slowdowns in GDI+ or WinForms, utilizing multithreading, synchronizing with an online (API) service, or are considering a rewrite because of C# performance issues.

Back when the Producteev Windows application was in the design phase. I knew right away it would require at least 2 threads to run smoothly. Producteev for Windows needs to synchronize with a Web API and making these slow API calls from the main thread would cause the interface to become unresponsive. Additionally, sometimes these calls may happen when the internet connection is unreliable, or the API service is offline, or the user is not connected to the internet. Therefore the synchronization engine must operate on a separate thread and handle all these conditions reliably.

“That’s a start,” I thought, and I ran with it.

The central idea was that all of the web calls would be serialized since order of operations is very important. For instance, consider the following user actions:

  1. Create a label
  2. Create a task
  3. Assign the label to a task

If the actions are sent to the web controller in reverse order, the instructions would be incoherent; you’d immediately trigger an error and not everything would be completed:

  1. Assign the label to a task (Error: No such task!)
  2. Create a task (OK)
  3. Create a label (OK)

Any change to model data needs to happen in a serialized fashion (i.e., in order, one at a time.) This is true both within the synchronization engine as well as locally to the program’s own data model. The latter would happen organically through user actions, and the former would have to use a dynamic circular buffer of sorts to line up HTTPS calls. This was the core requirement of the synchronization engine.

Read more about the Producteev Desktop synchronization engine design.

Bug reports

It was not immediately obvious why, but this design led to some unexpected behavior for some users, including:

  • Very slow application performance
  • Bulk actions are always slow
  • Crashes
  • Inconsistent state between desktop and web application
  • Interface hanging

The resulting behavior was contradictory to the intended design.

Astute readers may notice that local model data might have to be accessed by more than one thread simultaneously with this design. This is true; both threads did access the local model data, and that would cause symptoms similar to the ones listed above. However this is a red herring. All model data was modified and read with C#’s AsParallel() PLINQ methods and there were no noticeable race conditions or deadlocking as a result.

The problem was deeper

The reported issues were just the symptoms. Some testing revealed even more:

  • The interface would refresh multiple times after any update to the data model (e.g., after something simple like renaming a task or a workspace 100+ paint calls were being made)
  • The interface could theoretically refresh data that the user was actively modifying (such as a task title) which was rare but is an aggravating UX no-no
  • The program was slightly unresponsive, despite low resource usage

I fixed some of these directly by filtering the specific conditions from happening and releasing hotfixes. Assuaging the user experience issues like these is called symptomatic treatment since it doesn’t get to the root of the issue. The original problems remained.

My first insight

It began with investigating the performance issues. I said earlier that CPU and memory usage stayed low despite the application’s lack of responsiveness. I had initially attributed it to the C# language and its notoriously slow managed libraries. The entire interface is composed of controls that are hand-drawn in GDI+ onto WinForms. I made hefty optimizations to these and conceptually they were as fast as I could make them by using some of the same drawing techniques found in game design. Optimizations yielded marginal performance improvements, but I moved it on because the reliability issues were more disconcerting.

Before I continue, I want to note that it’s really easy to place the blame on the operating system, the language, or the libraries as being the cause of all of the above issues. Especially in this case, heavy drawing code and having to iterate through thousands of objects in a language that doesn’t approach the speed of C++ can easily explain away performance issues. But that’s not why the program was slow in my case. Let this be a lesson that you or I should never assume it right off the bat. Running into these issues and expecting a new language, tool, or a rewrite to fix it is often misguided — so Blame Yourself First.

The Culprit

It took a while but I tracked the issue down to being a central architectural issue.

Sometimes you will find that you make an invalid assumption when designing an application. Building layers upon layers based on an invalid assumption will usually cause problems down the road and they will usually be hard to fix. These problems may require some quick refactoring to fix it or they may be a killer that destroys the chances of success for a project. If your problem’s root lies in the core architecture it can spell disaster so take the time to carefully dissect the problem, consider its context, and focus on looking for neat, simple solutions.

The assumptions I made in this case was that the changes to the local data model would all be made serially, would be computationally trivial, and wouldn’t cause any noticeable delay in the interface. Here’s a simplification of the projected workflow:

  1. The user types in a new title for a task and hits enter
  2. The application directly updates the local data model to use the new title
  3. The application refreshes all views that contain the title to reflect the new title
  4. The application tells the API controller to add a new action to the synchronization engine to update the title
  5. The task’s title is now updated on the website in the background by the synchronization engine, leaving the main thread free again to handle additional input
  6. Once the web action is complete, the server returns the complete task object as a callback and I verify the change was completed successfully
  7. The returned task object is parsed and put into the local model by tying back into the main thread and updating the local model once more
  8. The application’s view is refreshed once more

Steps 1-5, 7 and 8 are done on the main thread, meaning during these steps the application cannot handle any input. Even during step 3, the calls to invalidate the main view are lined up in the Windows message queue, immediately deferring the WM_PAINT calls. (Quite a few of these can pile up quickly.) Bulk actions such as deleting twenty tasks will hold up the main thread while 20 local model updates 20 API controller calls are fired.

After some testing, I found that each of these steps are computationally trivial alone, but together lend to around 100ms of expensive interface lag for each action. This means updating 20 tasks might take a whole second for the interface to update even though we’re not waiting for the API calls to complete. This is well beyond the measure for being acceptable so it was taken into consideration when attempting a fix.

Resolving all of the issues

I chose a twofold approach that not only dealt with the performance issues, but also–and more importantly–dealt with synchronization reliability. This might not have been an ideal solution to start with, however it was the most elegant and cheapest solution I could implement:

  1. I created a local synchronization engine that works a lot like the remote one. It has its own separate thread and lines up any local model updates in order. This also means all local model updates are now piped through a single interface, clearing up the inconsistencies end users were seeing.
  2. Drawing is also on a separate thread now rather than using Invalidate(). Typically with GDI+ you will use Control.Invalidate() to redraw parts of a window. This lines up a new WM_PAINT message for the control which is processed during the main thread in the order that it was spawned. If dozens or hundreds of these line up like when you resize a Window, it can quickly flood the windows queue and make the application less responsive.

I recognize I could have also opted for implementing WPF from the beginning. This would enable hardware rendering to take the stress off the CPU and prevent a lot of the issues handling data from the model. Unfortunately at present there are so many WinForms and custom controls that it would’ve never been completed in a reasonable amount of time. Or ever.

Conclusion

I will reiterate my point earlier that this was more about resolving an issue the best way possible given the circumstances. For me this excludes the possibility of rewriting 10,000 lines of drawing code. The inconsistencies between local and remote (API) state are now resolved because all the updates to the local data model are done atomically and in a centralized way. And finally, threading the drawing code is a great way to prevent clogging up the main thread and the WM queue. The application is solid and responsive and it didn’t take a whole rewrite to do it.

Bricks Arcade Announcement, Updates

Posted on by

Bricks ArcadeI’m going to be releasing Bricks Arcade, an Android version of my desktop arcade game by June 1st.

There are also new details on my projects page to include engineering projects I’m working on. I have a lot of notes to parse through and compile for both the junkyard jet engine page and especially the more lofty VTVL one. I plan on making the latter a community-based project and will likely create a separate forum and software repository to allow interaction for it, considering the size.

As a side note it has been stated on electrical engineering StackExchange that hardware projects are notoriously less “open source” because of the amount of cost that goes into prototyping, the ubiquity of software compilers and tools in comparison to open circuit testing software, and so forth. I think it’s also a matter of fandom for the subject, which was high in the Silicon Valley days but dropped off in recent times. Don’t get me wrong, you still see people hand-wrapping home-made Apollo Guidance Computers, but with the advent of ICs the homebrew electronics crews just seem to vanish. It’s sad, being such a personal fan of hand-craftsmanship I want to see more of these projects thrive.

Tweaking AirTerm for Android

Posted on by
AirTerm Snapshot

AirTerm Snapshot Courtesy Google Play Store

AirTerm is a $2.99 app on Google Play that allows you to create floating terminals over any application. I usually use it to SSH into my server, light coding, or a quick nslookup and whois. The default configuration for AirTerm is somewhat unusual, though. For instance, it would report ‘Permission Denied’ when mistyping a command.

The following configuration changes assume that you have installed KBox from within AirTerm. The KBox installation provides a chroot, BusyBox binary, and a set of symlinks to BusyBox for basic commands like ssh and so forth. It also provides binaries for separate applications like vim.

You can change your command line shell. I set mine to:

/system/bin/sh

This will use your Android device’s default shell rather than the kbox_shell binary. The original kbox_shell is what causes ‘Permission Denied’ errors whenever you improperly type a command.

Now the next setting, “Initial Command” setting is found right underneath the shell setting. I have changed mine to this:

export PS1="$PWD$ "
export PATH=/system/bin/:/data/data/com.myboyfriendisageek.airterm/files/kbox/bin/:$PATH
export HOME=/sdcard/Documents
# insert any aliases here too
cd $HOME
clear

The Initial Command field supports multiple lines and outputs everything directly to the console. If you want something more dynamic (for instance, echoing a MOTD or system information,) consider dropping this into a shell script instead, and have the Initial Command field execute that script.

A quick breakdown: These lines add the system binaries and the kbox symbolic links to the system path (so you can still run commands like vim, ssh, and others), sets your home directory to your sdcard Documents directory, and clears the screen to a fresh clear state.

Works great! Now I just wish it supported X11 forwarding!

Bricks 10 Year Anniversary

Posted on by

Bricks

10 years ago today I publicly released my first PC game. It was my first jump into C++. Today I recompiled it in Visual Studio 2012 and aside from a couple of hiccups in the upgrade process it runs just as well as it did while I was in High School.

I think it has the neatest code I’ve ever written, but uses some defunct APIs and calls that shouldn’t be used anymore. As a result it probably doesn’t handle Unicode very well.

Download Bricks (.zip)

Unzip it into a folder and double-click the executable. It spits out some files in its working directory for settings and high scores and such.

Working on Linux, Week 1

Posted on by

Yesterday marked my first full week of using Ubuntu Linux 12.10 at work. In short: Well naturally, I totally love it.

Is Linux For You?

It’s excellent for every-day PC users, perfect for web workers, developers and engineers, but it would fall short for most graphic designers[1].

A quick search reveals that you may get most common audio and video production tools running under Linux. I’d guess that it would be clunky and not worth it for most. Specialized software that was designed to run exclusively on Windows will usually run into noticeable performance issues in a virtualized environment.

On the upside, writers would be sated by Linux’s wide array of free writing and typesetting software. Engineers also have an array of CAD and electrical design automation (circuit design) applications. These tools go outside the scope of this article, but I recommend you look further into them anyway as interesting and viable alternatives.

Linux gamers aren’t being ignored any longer either. Steam is working on porting their software to Linux. Humble Indie Bundle also famously supports Linux. There’s a number of cool smaller games you can get from the Ubuntu Software Center too for casual players. For hardcore gamers, Play On Linux has a compatibility guide to see which AAA Windows titles will run smoothly on Linux using their software.

Logistics aside–if you’re a person who’s especially passionate about the inner workings of computers, Linux is a great way to learn more. Conversely, if you want to take more of a hands-off approach and just want a secure and reliable computer to check email, read news, and write documents on then Linux is equally suitable.

I am looking forward to expanding on this more in the future because I have quite a few tools that I want to share.

Footnotes

1. Disclaimer: This is assuming most designers are going to be using Adobe products. Adobe and GIMP do not inter-operate very well from my experience, so there are instances where you may not have a choice. However, if you work better with GIMP and Inkscape than Photoshop and Illustrator then more power to you; you’re using software that has similar features and you don’t have to pay $600 for it! More on this later, though.

Back to top

The Status of the Producteev Windows Client

Posted on by

At the time of this writing, there are over 41,000 installations for the Producteev Windows Desktop application. That number is pulled from real aggregated server data and seems way too large to be real. Assuredly, it is.

I was the sole developer on this particular project, living and working in NY for a year. Producteev has an intuitive team that gave me all the support I needed to make it work. Right now that team is transitioning from New York to California following their recent acquisition by Jive. A huge success, that was built by solid software and a smart people.

I digress; back to the desktop app. Here’s where it stands now:

  • It suffers critical issues (in particular, syncing) for a portion of users
  • It’s missing features that the main products all have
  • The erratic “edge-case” behavior is difficult to diagnose
  • There are performance issues relating to drawing and the Windows message queue despite optimization

The Windows application still gets a lot of regular use from clients all over the world. Those users deserve to have software that works really well.

The most critical issues stem from:

  • Improper unit testing (my bad)
  • Leaky abstractions from bad parsing and database libraries
  • Database format
  • Inconsistent API results, which have mostly been fixed
  • Slow C# GDI+ calls instead of using hardware accelerated WPF
  • Large .NET client requirements
  • Multithreading and inefficient delegates make updating the view a risky nightmare

I would like to say that this doesn’t account for 90% of all of the code, but it definitely does. The enormous requirements for an overhaul make it a good candidate for porting. The project is presently on hold pending some more testing.

Keep an eye out for more info.

Officially Switched to Ubuntu 12.10

Posted on by

I’ve officially dropped the axe on Windows for good. In the past Linux seemed to be too unstable, but after evaluating a full install of Ubuntu Linux 12.10 I’ve determined that it works beautifully and much more effectively than Windows 7 ever did. (As a side note I also use a Mac at work and I’m a big fan of them too.)

Why?

Ubuntu works well for every day uses: editing documents, spreadsheets, and presentations, securely browsing the web, checking email, chatting, playing web games, and so on. For an average computer user, you’ll find it has everything that you need without having to install additional software.

Security for the platform is unparalleled, and you can expect to never run into viruses and malware as one typically would on a Windows computer.

Hardcore gamers will also be delighted that Steam is preparing to launch on Linux soon.

Getting It

If you want to make a bootable USB drive to install Linux, I recommend you check out unetbootin. Aside from saving you the CD, it’s also faster to run off of and a great way to evaluate it before you install. Unetbootin is available for Windows, Mac and Linux. If you are coming from Windows, Ubuntu 12.10 is extremely user-friendly, refined, and works great out of the box. You can also install new software really easily by using the Ubuntu Software Center.

12.10 is not available for unetbootin by default, but you can download the iso from http://www.ubuntu.com and then open it from unetbootin.

Running Windows Applications

If you still use proprietary software for Windows, you can usually use Wine to run them under Linux seamlessly. Wine can be installed easily through the Ubuntu Software center with just a few clicks.

An Important Note About Corona SDK

Posted on by

I’m posting this because hopefully it may save a few people out there some time and frustration:

If you specify a scale mode in your config.lua file, you will no longer be able to get physical dimensions of the screen.

The reason for this is because if you set scale to anything other than “none” you’re not working with direct screen coordinates, but rather relative coordinates within a viewport that you’ve now specified. In other words: regardless of the devices resolution, you will always be working with the width and height you specify.

For games, you might want to thoroughly consider which zoom mode you want to select.

  • letterbox – Useful only if you can bleed your background content beyond the normal viewport stage. Don’t ever release a game or app with black letterbox bars for everyone’s sake; follow Corona’s best practices to fill the black letterbox regions.
  • zoomEven – This zooms the viewport until the entire screen is covered. This means you can almost always count on part of the viewport getting cut off. Don’t put text or interactive elements near the edges of the viewport that you’re working with because they will not be seen on some devices.
  • zoomStretch – Absolutely useless garbage. Don’t even consider it. What is this even for?
  • none – This is your true 1:1 mode. You have to consider this mode very carefully because on higher-resolution devices your elements may show up smaller. However, if you have a heads-up display or visual elements that need to be at the corner of the screen for usability, then this is your only true option.

Check out this post for more information:
http://www.coronalabs.com/blog/2010/11/20/content-scaling-made-easy/