Founder of Kernl.us. Working and living in Raleigh, NC. I manage a team of software engineers and wrk in Python, Django, TypeScript, Node.js, React+Redux, Angular, and PHP. I enjoy hanging out with my wife and son, lifting weights, and advancing Kernl.us in my free time.
In part 1 of my “Learning Clojure” series, I created a simple program to calculate salary based on how many years someone worked. For this post, I’m going to be attempting something a bit more complicated.
One of my favorite websites in the entire world is Project Gutenberg(PG). PG is an archive of books that have passed into the public domain, which makes is a great resource for text mining data. I use it almost every time I need some words to parse, and you should too! So why does this matter right now? I’m glad that you asked.
Given how simple the last program was, I decided that I should probably take this one up a notch. Its going to involve fetching a file, writing it to disk, reading the file, and processing command line args. In order, here’s what the program needs to do:
Validate command line args – We’re going to accept two arguments. The word that should be counted and a url that points to a .txt file at Project Gutenberg my web host (Project Gutenberg doesn’t like crawlers apparently) for processing.
Download the file – It could be large and might fail. We’ll need to be careful here.
Split the file into a vector – Split the file up on ” ” and load it into a vector.
Print – Print to standard output how many words were found. If none, make it known.
Clojure is a functional programming language based on Lisp and written to run on top of the JVM. I’ve tried learning it in the past, but have failed mostly due to biting off more than I could chew. But not this time! I’m taking my time, reading lots of code, and doing 1st year computer science assignments with it. I figure this worked well when I first learned how to program, so it will probably work well now.
The Return to Trivial Programs
After spending the past 6 years neck deep in non-trivial professional programming, I’m returning to trivial toy programs to learn Clojure. My first task is to write a program that takes user input from the terminal and calculates their salary at a year which they input. More specifically:
Starting salary is $1000
Salary doubles every year
Validate input to make sure it is a number.
Write history to file called: salary_history.txt
In format…. [years_working]:$[salary]
"Returns a string in integer form, else false."[input](try(#(Integer/parseInt %) input)(catch Exception e false)));; Incomplete. Will eventually write to a file.(defn output
"Takes the console input and error message and outputs them to file and console."[console-input message](println (str console-input ": " message)));;;; ????? WTF DO I DO HERE;;(defn calculate-salary
[& args](println "How many years do you want to work?")(let[user-input (read-line)](let[years (get-integer user-input)](if years
(calculate-salary (- years 1)1000)(output user-input "This is NOT an integer.")))))
"Returns a string in integer form, else false."
(#(Integer/parseInt %) input)
(catch Exception e false)))
;; Incomplete. Will eventually write to a file.
"Takes the console input and error message and outputs them to file and console."
(println (str console-input ": " message)))
;; ????? WTF DO I DO HERE
(println "How many years do you want to work?")
(let [user-input (read-line)]
(let [years (get-integer user-input)]
(calculate-salary (- years 1) 1000)
(output user-input "This is NOT an integer.")))))
The Python implementation of calculate salary would look something like this:
salary = 1000
for i in range(years-1):
salary = salary * 2
But in Clojure things are bit more complicated. In Clojure values are immutable. I can’t just loop over the years and keep doubling the salary while storing it in the same variable. I need to use recursion. Or reduce. Or map. Hell, I don’t know. I need to use something functional, lest I want the Clojure experts to laugh at me. I need something that will call a function that doubles whatever value comes into it, then returns. Then I need to call said function up to N times (where N is the number of years that the person enters).
With the help of Ryan (below), I came up with:
[years salary](if(= years 0)
(calculate-salary (- years 1)(* salary 2))))
(if (= years 0)
(calculate-salary (- years 1) (* salary 2))))
Over the past few years I’ve slowly accumulated some big side projects. They weren’t done for clients, but just for myself. At some point maintenance of these side projects isn’t fun anymore and hinders the creative juices. I have other things I want to work on, but having these other zombie side projects feels too much like an albatross around my neck.
After much deliberation, I’ve decided to shut down two of my large side projects: BookCheaply and Smooth Bulletin. I really believe both of these projects could do someone some good, but they were both learning projects for me and I don’t see them moving forward anymore. Effective immediately I’m disabling their Apache configs, backing up their DBs, TARing it all together, and putting it somewhere safe. I’ll keep access to the Git repos, but eventually I’ll clone a copy of those out too and archive them. If I don’t get them out of the way completely, I feel like I’ll want to work on them too much.
Shuttering these projects marks a transition for me, where I move from using Python and Django on side projects to Node.js, Express, and Angular. While apprehensive about abandoning my go-to stack for side projects, I’m excited to learn the nooks and crannies of Node (and I still use Python/Django for my day job anyways).
A long time ago in an era just before the first dot-com bust, I was a bright-eyed child with aspirations to make the greatest Star Wars web site of all time. With only Microsoft Front Page Express and a limited knowledge of HTML in hand, the 1999 version of me created the best Star Wars web site the world had ever seen.
Unfortunately I was hosting it on my ISPs free web hosting area. When that ISP was later gobbled up by another company, all traces of my website were gone… or so I thought. Recently I discovered that most of it still exists on the Way Back machine, however it’s in poor shape. Many of the assets don’t exist anymore, the linking is broke, and the frames just don’t work right in modern browsers. This is my journey to save this little piece of history while using minimal modern techniques. My goal is to have a legitimate 1999 website when I’m done and have it render nicely on most modern web browsers.
The Internet Archive
The Wayback machine at the Internet Archive has long been one of my favorite websites. I love getting nostalgic for the web of the past, my childhood, and history in general. It was awhile ago that I realized it still had some of my original Star Wars website. If you want, check out the capture from 2001. It doesn’t work much at all, but it gives you a good idea of what we’re working with here.
In addition to the main page I also had an “Entrance” page, because you know, every cool web site had an entrance page back then. This one is in slightly better shape, meaning the star field background survived intact.
The Game Plan
Now that we’ve laid down some ground rules, here is the order of attack:
Entrance: Fix / replace broken images.
Entrance: Clean up the HTML (alignment) and remove anything that the Internet Archive crawler added.
Main site: Fix / replace broken images
Main site: Remove dead links. Sadly the Internet Archive couldn’t capture everything.
Main site: Fix any styling issues that remain.
Main Site: Clean up HTML (alignment) and remove anything that the Internet Archive crawler added.
Lets get started!
Entrance: Replacing Broken Images
While it’s fantastic that the background star field has survived, the rest of the images didn’t. This is sort of a problem because I have to reach pretty far back into my memory to figure out what was there originally. Luckily, 1999 me wasn’t completely terrible at naming things.
From these file names and memory, I’m going to say that `tie.gif` was a rotating TIE fighter gif, and `starwars.gif` was the Star Wars logo that survived on the main section of the site.
And from this file name we get nothing. Luckily I remember this being an image of Tatooine, roughly 350px x 100px. This site came out a bit before Episode I, so I’m going to assume it was an image from one of the Episode I trailers. At least, thats how I remember it.
After a bit of digging, I found replacement images for everything but the Tatooine banner. In it’s place, I found a light saber gif that I definitely used on this site somewhere in the past. I also updated the star field to be a little more 1999.
Entrance: HTML Clean Up
Now that the entrance is starting to look like a real 1999 web site again, we can start to clean up the HTML and fix the broken link to enter the site. The entrance area contains 3 pages, 1 to bring the frames together, and 1 for each frame. They currently look like this:
The code isn’t too bad. With a bit of alignment help and some nesting, things are going to look pretty good. Now check out the final product:
Main Site: Fix Broken Images
Now that we have the entrance out of the way, I’ve have a pretty good idea of how this whole restoration will work for the rest of the site. Let’s start by fixing the background on the main site so things are a bit more workable.
Now we’re talking! There are few things that I noticed immediately after getting the background set.
1999 me had some pretty sweet design skills
I should use more animated gifs on my websites
The frame content on the left isn’t staying inside of it correctly, so I had to scroll to the center to get it to work.
I had a Yahoo! club!
I wasn’t so great with words back then. (I was a kid, give me a break)
Looking at the source, there are a few images that aren’t linking quite right.
Left side: starwars.gif – I think this was just a smaller version of the big logo.
Left side: deathstar.gif – Definitely a gif of the Death Star. It might have been animated, but I think it was just transparent so that it looked awesome.
Right side: lettersabove.gif – I honestly don’t know, but it might have been “A long time ago in a galaxy far far away…”
Right side: pulselightsaber.gif – A pulsing light saber page break. I might just use the static one from the entrance instead.
Right side: xwing.gif – A rotating X-Wing fighter. Because nothing says “Email me!” like a rotating X-Wing.
After tracking down many of the original images that I used here, the home page now looks like this.
Looking good! Next up, code clean up.
Main Site: Code Clean Up
The code for the main site is in pretty poor condition. There is a ton of code in there from the Internet Archive, it has poor indentation, and in general just isn’t anywhere near the quality of the entrance. Given the amount of code here, I thought using a Github Gist would be better.
The next step in the code cleanup is fixing URLs. If you look at the code, you’ll notice that all the links still point to the Internet Archive. That obviously isn’t going to work for us, so I simply need to replace “https://web.archive.org/web/20010811043323/http://my.voyager.net/~hands/star-wars/” with an empty string, making the link relative.
After fixing links, I found that many of the sub-pages have been lost to time, but there are still a few for us to look at including:
G.Moff Willguf Tarkin
Star Wars: Rogue Squadron
Mon Calamari Cruiser
Fixing up these pages follows a similar pattern to the entrance page and the main site, so I’ll spare you the details. There were a few interesting things that popped up while I was doing the sub-page restoration though.
Plagiarism – 1999 me definitely ripped off somebody else with most of the encyclopedic data on characters, weapons, and ships. The change in voice is a dead giveaway.
Front Page Versions – It appears that I made the entrance and the menu/main page of the main site in FrontPage 3.0, but all of the sub-pages in FrontPage 2.0 (express). FrontPage Express was shareware that came bundled with IE 4 so I know how I got that. I believe I had access to FrontPage 3.0 at my school at that time, which I probably used to generate the frames.
Xoom – I had completely forgotten about it, but Xoom used to be a free unlimited web host back in the early dot-com days. I used it to host videos, games, and other things. Unfortunately my Xoom site wasn’t crawled by the Internet Archive.
Java Chat – Back in 1999, there weren’t a whole lot of options for getting a chat room going on your website. Using a Java applet was basically it, so thats what I did. It looks like Xoom offered it’s members a free chat tool, so how could I resist?
Game Demo – Ah, the good old days. The one game page that survived has some pretty hefty system requirements: Windows 95/98, 32 MB of RAM, Pentium 166. Direct3D card, DirectX 6. Oh, and I also made sure to let people know the estimated download time on 28.8 kbps dial-up modem.
While going through this restoration I realized that this is when I knew what I wanted to do with my life. My parents had no idea what I was doing, but they realized that it stimulated my mind so they let me continue to do it anyways. Without that kind of encouragement and freedom to create, I probably wouldn’t be where I am today. This restoration has also re-kindled my respect for the old web. It was simple, but I’m extremely happy that the web has progressed to where it is today.
Ever since HealthCare.gov launched all I’ve been hearing about is how horrible the experience is. It never affected me directly, so I chalked it to a few vocal detractors trying to sway public opinion. I even went so far as to go to HealthCare.gov and use the logged-out experience to compare some plans. It looked pretty good actually, and I tweeted as such.
That was all about to change though. I recently made the move to work at a small startup that doesn’t yet have an employer-sponsored health plan. They do however offer a stipend of sorts to help you pay for insurance costs on HealthCare.gov. And with that, my journey began.
Initial Sign Up
After discussing the healthcare situation with my wife, I ran the numbers and decided that it would be best for her to use her employer-sponsored healthcare and for me to use the exchange separately. At this point everything went as well as it could. Even though the UX on HealthCare.gov is pretty bad, I was able to figure out how to fill out an application, select health coverage, select dental coverage, and get things rolling. This all happened before December 23rd, which means I’d be getting coverage by January 1st! All was well… until I had to make a change.
As it turns out, I math’d wrong. When I was calculating the cost of healthcare for my wife at her job, I was off by something like $200 per month, which pushed her healthcare costs well into the “unreasonable for what I’m going to receive” range. With that in mind, I thought I could make a quick change on HealthCare.gov to get her on the same plan that I was. Boy was I wrong.
After finally getting through the application process, I get to the last step where I need to confirm. I press the button and… nothing. Not a goddamn thing. Being a software engineer, I do a little investigating and find out that the server is returning 500 errors to the client (in non-developer speak, this means that the server couldn’t process my request because of some error on their side). The error that is returned says that I should try to log-out, then resume the application. I do this, but when I try to resume my application I get:
Thats right, my application is locked. It doesn’t give any reason, which is especially confusing considering that I was asked to re-authenticate. But there is a little link that can “explain this task”, so I click it, and it naturally didn’t work.
At this point, I was beginning to realize that I would need to contact their support to get things resolved.
Support and Next Steps
The first time I tried to use support, I clicked the helpful little “Live Chat” button. I then proceeded to wait for 25 minutes with no response. This was over the holidays, so I gave it another try after Christmas. This time I was connected to somebody, but no matter what question I asked I was given a canned response to contact the call center. My question is: Why have the “Live Chat” at all if you’re just going to tell me to contact the call center? It’s ridiculous and a waste of my time.
After my initial experience with HealthCare.gov, I’m not really excited to contact the call center. I honestly don’t have a ton of time to do it, and given the number of people that will be signing up for healthcare I’m sure the wait will be long. My likely next steps will be to cancel what remains of my application and start over. If it doesn’t work this time, then I’m going to cancel that application and contact my insurance provider of choice directly. In my case thats Blue Cross Blue Shield of Michigan, which happens to have a walk-in office about a block from where I work.
I create things all the time on the web. Its my chosen profession, so I know how hard it can be to make a good website when you need to integrate with a lot of different 3rd parties. However this sort of experience isn’t going to cut it. If I’m federally mandated to have health insurance, then the experience should be as painless as possible.
Because the system is broke I won’t have health coverage until February 1st now; a month long gap in coverage for my family. This just isn’t acceptable.
If you browse Hacker News enough you’ll start to see article trends.
Startup X is Startup Y for Dogs
Why you should use framework Z over formerly hot framework B
How I failed at A and it made me better at B.
Don’t get me wrong, some of this stuff is interesting, but most of it is just noise to me. The reason I come to HN is for the comments. Thats where the good stuff is. HN is one of the brightest internet communities I’ve ever had the pleasure to interact with. For instance, today I posted to HN asking for the community to review my startup Smooth Bulletin. Within an hour I had two very thoughtful comments about business and market ideas that I hadn’t even thought of. I expect I’ll probably get even more feedback as the day goes on, and that I’ll benefit from those just as much as the first comments.
Thats why I still have faith in Hacker News. Not because of the content, but because of the people. The people there are incredibly smart and willing to give their advice just so that maybe you can succeed one day. I’ve been a member of HN for several years now and this hasn’t changed at all since the time I joined, and I hope that it never does.
And yes, I posted this to Hacker news, so maybe I should add “Why I still have faith in X” to the list.
TastyPie’s JSON responses split the response into two sections: meta data and actual data. The meta is really nice, because it helps you data pagination, result counts, and the like, but it kind of gets in the way of Restangular. Integrating with Restangular is easy though!
For a few hours I was running into a problem whenever I would try to install my PIP requirements file. The install would go alright until it got to Distribute, at which point I would get an that ended up in a stack trace with:
Could not find the /lib/python2.7/site-packages/site.py element of the Setuptools distribution
After much searching and digging, the issue was that my virtual environment needed to be instantiated with the –distribute flag.