zip and unzip in linux

So with all this migrating that I’ve been doing, I’ve come to realize that it takes an excruciatingly long time to download thousands of files and then upload them somewhere else, even if the total filesize is relatively small. Uploading a basic WordPress install can take 10 minutes, and it’s only about 3MB worth of files. That SHOULD only take you, depending on your connection, about 10-20 seconds.

The reason it takes so long is because every time each of those files is uploaded, your FTP client has to send information to the server to connect, then upload the file, then send more information to close the connection. All of the connecting/disconnecting adds a significant amount of bandwidth and time overhead.

The solution? Upload a single file. If you were to directly upload the zipped WordPress package, it would take only 10 seconds. And if you have shell access to your server and it has zip/unzip installed on it, then you can extract the contents directly on the server. Even the extraction only takes seconds.

Again, like my post about transfering website from one server to another, I can never remember the proper command to zip or unzip a directory. There are a million different things you can do with the zip command, but I have no idea what any of them do, and they just confuse me. I figure I don’t need to be too concerned about the compression type or backup tar whatever-you-call-its when I’m just uploading photos of my dogs to my blog.

So here’s the zip command:

zip -r [output filename] [directory name to compress]

NOTE: The output filename does not need to include the file extension (i.e. .zip). Also, I think it’s best to cd into the parent directory before issuing this command. The directory name could potentially be entered as a path, but I doubt it. Not worth testing. And lastly, the -r only needs to be included if compressing an entire directory (it stands for “recursive” meaning it’ll include all files and folders inside of the specified directory). You can leave it out if compressing a single file. And lastly lastly, don’t be a bonehead and include the square brackets. I just used those as a visual cue.

And as for unzippng, the command is simply:

unzip [filename]

This time, include the .zip extension.

That’s it!

No Comments »
date Jan 28th 2011
author Mike
category Geek
tags(none)
 

migrate website from HostMonster to Linode

I created this blog for three reasons, which are reflected in the three main site categories: geek, life, and photo. The geek category was supposed to be a place where I could talk about computer and internet stuff, and to make tutorials for things I struggled with and thought “there should really be a tutorial explaining how to do this.” I’ve never actually created a tutorial, so here goes my first attempt.

This is a checklist of things I had to do to move one of my websites (mikeneedsajob.com) from my HostMonster web server to my new Linode one. I’ve outgrown HostMonster in the last two years that I’ve been with them, and I decided that it was time for me to learn a little more about hosting. HostMonster puts a very thick layer between you and your server, making every task easily accomplished though a user-friendly interface. Linode takes all of that away, but still holds you hand as much as you need while setting up your own server environment, making it the perfect learning ground.

I spent all of today figure out this process, so here it is, from beginning to end. Hopefully this will save me a headache in 6 months when I might have to do this again and I forget one of these steps.

NOTE: These steps only apply to my specific Linode setup, which could very easily differ from yours. I’m running Ubuntu 10.04 LTS, Apache 2.x, PHP 5.x, and MySql 5.x.

transfer files

Step 1 is to transfer all of the site files from your HostMonster server to your Linode one.

This is simple, and you should probably already know how to do this. Using simple FTP to move the files to-and-from your own computer should suffice, unless your site is extraordinarily large. When you upload to Linode, you should be putting your websites into the /srv/www/ directory. Now, if you plan on having multiple websites hosted on this server and want to keep things clean (a concept you’re probably not familiar with since you’ve been using HostMonster), create different directories inside of /srv/www/ for each site. Personally, I like to be overly organized, so here’s the file structure I used:

/srv/www/mikneedsajob.com/v1/public_html/

I dumped all of my files inside of public_html. If your site uses scripts that shouldn’t be publicly available, you can put those in an includes folder in the v1 directory.

As a side note, the v1 is because several sites I manage have older outdated versions, and I like to keep records of them. Ultimately, this v1 directory might become a subdomain of Mike Needs A Job v2.0 for posterity’s sake.

transfer databases

If your site uses any MySql databases, you’ll need to transfer those over, too. Before you start the import, you’ll need to create empty databases on your Linode with the same names as the ones you’re going to be importing. I don’t have phpMyAdmin installed yet, so I SSHed into it (I’m assuming you know how to do this since it’s required for you to initially set up your Linode), logged into MySql, and created the database by issuing this command:

CREATE DATABASE [db name];

Then in HostMonster, log into the cPanel, then into phpMyAdmin and create a .sql export of your databases. Once you have it downloaded onto your computer, FTP it up onto your Linode. It doesn’t matter where you put the file on the server, but make note of the location. To make things easy, I simply uploaded it to the v1 directory inside of mikeneedsajob.com.

Once it’s uploaded, SSH into your Linode again, and issue the following command from the shell:

mysql -u [username] -p[password] [database-name] < [path-to-mysqldump-file]

Replace everything in square brackets with the appropriate information. Make sure there is no space between the -p and your mysql password (it looks wrong when you’re entering it, but trust me). If my .sql file was called dump.sql, the path for me would be /srv/www/mikneedsajob.com/v1/dump.sql. Log into mysql again and check that everything got imported correctly. To do a quick check that all of the tables exist, type this command into mysql:

USE [db name]; SHOW TABLES;

prepare apache

Now that your entire website data has been copied to Linode, you need to prepare Linode to serve the site. First we need to tell Apache what to do with incoming requests for the site. I’m going to assume you’ve already set up Apache for virtual hosting (if not, follow the steps here. This whole section actually just comes straight from that guide.). First we’ll create a file in the /etc/apache2/sites-available/ directory named after the website (i.e. mikeneedasjob.com). To do this, cd into the sites-available directory in the Linode shell, and then issue this command:

vi mikeneedsajob.com

This will bring up the vi editor for a new file named mikeneedsajob.com. Go to that link for basic vi commands. To enter text into the new file, hit your i key, and it should bring up a cursor on line 1. Then paste or type the following text into it:

<VirtualHost [your ip address]:80>
     ServerAdmin example@mikeneedsajob.com
     ServerName mikeneedsajob.com
     ServerAlias www.mikeneedsajob.com
     DocumentRoot /srv/www/mikeneedsajob.com/v1/public_html/
     ErrorLog /srv/www/mikneedsajob.com/logs/error.log
     CustomLog /srv/www/mikeneedsajob.com/logs/access.log combined
</VirtualHost> 

The ErrorLog and CustomLog lines are optional, and if you choose to enter them you’ll need to create the logs directory. When you’ve entered that text, hit your esc key, then type :wq (which stands for write and quit). When back in the Linode shell, issue this command:

a2ensite mikeneedsajob.com

And to finish, restart Apache:

/etc/init.d/apache2 reload

add site to linode dns manager

Thankfully, Linode makes managing your site’s DNS terribly simple. Just log into manager.linode.com, click on DNS Manager at the top, and Add a Domain Zone. Tell them what domain you want to manage, and allow them to automatically add a few records for you. What results is the cleanest DNS manager with all the appropriate fields for you to add your A, MX, and CNAME Records, to name a few. You can go to this page for a little bit more help if you need it.

change dns nameservers

Now you’re finally ready for your website to be served from Linode. All that you need to do is point your DNS Nameservers to Linode’s.

If you bought your domain from HostMonster themselves, you can change the nameservers in the Domain Manager tab at the top of your account. If the domain is an addon to your account, highlight the appropriate domain, and disassociate it from your primary account. If it is your primary domain, you’ll need to do some clicking around yourself, because I have no idea how disassociate it. mikeneedsajob.com was an addon, that’s all the experience I have…

Anyway, once it’s disassociated, there’s an option in the Name Servers tab where you can choose to use custom nameservers. Select this option, and enter Linode’s nameservers.

If your domain is managed by another registrar other than HostMonster (e.g. godaddy, enom, etc), log into that registrar . Right now your nameservers are pointing to HostMonser, and look something like ns1.hostmonster.com and ns2.hostmonster.com. You’ll need to delete those, and add Linodes. Currently, they look like this:

ns1.linode.com
ns2.linode.com
ns3.linode.com
ns4.linode.com
ns5.linode.com

They should never change from those listed, but just in case, you might want to check what their current ones are.

All DNS managers tell you that nameserver changes will take 24 – 48 hours to take effect, but I’ve actually never seen them take more than an hour or two. I can’t promise that’ll always be the case for you, though. If you want to check that your nameserver switch has taken effect yet, look up your site’s whois info. What I’ve found to be the quickest and easiest way is to type whois mikeneedsajob.com into a *nix command line, and the nameservers will be listed right there. If you’re on a Mac, just type that right into the terminal window. Otherwise, your Linode is a Linux server, remember, so you can just SSH onto it, and type it into the SSH window.

Once your whois is showing that your nameservers are now pointing to Linode, then your site should now be served from your Linode. Hooray! To double check that you’re in fact seeing your Linode-hosted version, FTP into your Linode and change your index file slightly (add an extra period somewhere on the site). If you see the change on the live site, then you know you’re live. If you’re like me and ran this little test the second after whois showed your nameservers were changed, don’t be distraught if you don’t see your change yet. For some reason, it took my site a few minutes to fully realize where it’s new home was.

finishing touches

TEST! Click around your site, make sure nothing is broken. Going from one version of PHP to another might cause problems. Having a new file structure on your server could definitely cause problems, particularly if you ever reference files or directories by their full server path. Grep the site for your old server structure. To be honest, I’m a novice when it comes to using grep for it’s full potential, so I just use this structure and it always seems to work:

grep -lr "some string to search for" *

This will run a search inside of the directory you’re currently in and all of its child directories (so you should probably cd into your v1 directory before running the search). That particular query will only look for text inside of the files, it will not search filenames, directory names, or metadata. Your search string can be something like “public_html” or “usr” if you’re looking for instances of file structure mentions.

potential problems

I’ll add more problems here if I run into more as I finish migrating the rest of my sites, but so far this is the only one I’ve run into.

Mike Needs A Job had a few vanity URL rewrites, and various other .htaccess wizardry, and none of it was working after the migration. I also was testing the migration of a WordPress site, and its custom permalink structure was all sorts of screwy for the same reason. After lots of searching and troubleshooting, I figured out that Apache doesn’t have mod_rewrite enabled by default. I followed Linode’s suggested Apache install and setup procedures and it never said anything about mod_rewrite, so I’m guessing this is the case for everybody when they first get set up.

To fix it, I followed the instructions on this page. Apache’s available modules are in the /etc/apache2/mods-available/ directory, and in it there should be a file named something like rewrite.load. To enable it, type:

a2enmod rewrite

You don’t need to type the extension in when doing the enable command. Just to be sure the enable will take effect, go ahead and restart apache (remember, type /etc/init.d/apache2 reload). If you don’t see the rewrite.load file in the mods-available directory, then it might already be enabled which you can check by going into /etc/apache/mods-enabled/.


Well, I hoped all of this helped somebody. Let me know if you have any questions or need clarification anywhere. And good luck!

4 Comments »
date Jan 3rd 2011
author Mike
category Geek
tags,
  4 Comments

Make

prompt

What was the last thing you made? What materials did you use? Is there something you want to make, but you need to clear some time for it? #

response

I help make the internet. The last bit of it that I made was I Am The, a ripoff of another site I saw and thought sucked. I went into it knowing that it involved a number of problems that I didn’t know how to solve, and I was able to figure them out. I was pretty proud of it.

My next project is about 20 times as large, 20 times as complicated, and I have even less idea about how to implement it. Last night I bought a new domain name* to test it out on, and new server space over at Linode. Setting up that server alone was an incredible accomplishment for me (I’ve always been with HostMonster and only used their cPanel interface to set everything up…I’m ashamed to say that I know very little about webhosting), but I can’t take full credit for it; Linode has so many tutorials and step-by-step guides that they essentially hold your hand through the setup process. I’ve never used the command line cumulatively as much as I did last night. It feels like such an amazing accomplishment when you set up your own server, install your own LAMP on it, and see Apache’s default “It’s working!” page live on the internet. It’s an even bigger sense of accomplishment when you point your domain’s nameservers to your new IP and see it rendering the content you just added via SSH.

Holy balls, I geeked out hard last night.

* All 4 letter (not character) dot coms were taken in 2007, and there’s a good chance that somewhere near 99% of all 4 character (includes hyphens and numbers) are taken. My new domain is a 4 character dot com, and it isn’t gibberish. It has a 3 letter word followed by a number. And I only payed $10 for it. I can’t even tell you how stoked I am for getting that. Somebody had previously owned it, and let it expire…yesterday. I found it on an expired domain names list, and immediately snatched it up. Hell yes.

No Comments »
date Dec 8th 2010
author Mike
category Geek, Life
tags
 

html5

There’s been so much hype lately about HTML5 and how it’s the future of web technology standards. You can’t go to any major web browser’s website without being told about how awesome their HTML5 support is (Apple SafariMozilla Firefox 4Microsoft IE9). I couldn’t find a similar page on Google about Chrome’s support, but we all know about Chrome Experiments. It’s like HTML5 has become a marketing tool for companies to gain an edge over their competitors. But it hasn’t even been officially released or even suggested by the W3C (the web standards governing body), so I keep on asking myself “well, what the hell is it exactly?”

Good question, me. Well, Google claims that all of their mobile web apps are built using HTML5. Awesome, so it makes links more clickable using your thumb instead of a mouse pointer? Or is it actually the programming behind the apps? I mean, Google Voice is pretty slick on the iPhone, but what exactly does html5 do that made that app possible? With some clever browser detection and mobile-targeted stylesheets, I could probably create a UI that at least looks and acts like GV.

To try to answer my question, I started doing a good amount of research. I followed along with a bunch of HTML5 tutorials and created my “very own first website using html5″. A few months ago I even joined some fellow geeks at a presentation by Imulus all about HTML5. And still, I didn’t have a better understanding of what it was. In fact, most of the cool functionality that’s often attributed to HTML5 was actually CSS3 and JavaScript trickery, not some revolutionary markup language. And I’m not the only one that was confused. Apple has a set of HTML5 demos meant to show off Safari’s support, but only 2 of the 6 demos actually use HTML5.

This morning I gained some understanding for the first time. I started watching Lynda’s first look at HTML5 (alternately) video series, and it finally shed some light on the subject that’s confused me for so long.

Basically, what it breaks down to is that I was correct. Many of HTML5′s most touted capabilities actually have nothing to do with the language. Let me reiterate (this is a very important point): HTML5 has little or nothing to do with all the awesome “HTML5″ websites out there. The term itself has become a brand, or as I mentioned earlier, a marketing tool. It’s the new “it is the future” key phrase like “Web 2.0″ was a few years ago, and it’s become the latest hope for the loyal Apple user/Flash hater. Anything new and cool on the web is allowed to be placed under the the HTML5 heading. It’s not a web language anymore, but a shift in how users interact with the web, and how the web interacts with you.

The most important point I think the Lynda videos made was that it’s not so important for the end user to know the difference between what HTML5 is and what it isn’t (do you care about the differences between PHP and JavaScript? They’re vastly different, but the end user doesn’t need to concern themselves with that), but it’s monumentally important for developers to understand that difference. I was doing research as a developer, but finding results intended for the end user. I thought that HTML5 was the answer to every problem I’ve ever had with cross-browser compatibilities, embedding various media types, etc., when in reality it’s just a new way to markup pages (and in reality, more than half of its official specifications are accounting for existing markup) with a few new APIs (including native video and audio support).

I’m excited about HTML5′s current support and it’s imminent official release, but I’m even more excited to figure out how to create the falsely labeled “HTML5″ web apps out there. HTML5 largely leans on JavaScript to create most of its dynamic and interactive content, so this research has actually made me more excited to delve into JS than anything else. One of my biggest fears as a developer is getting left in the dust while clinging to depreciated web technologies, so there’s no way I’ll let HTML5 slip through my fingers. It is a part of the future, but it’s only one of the players. It alone can’t create the richly interactive websites of tomorrow.

No Comments »
date Nov 14th 2010
author Mike
category Geek
tags
 

incessant record keeping

So one of the biggest reasons I want a blog is to keep a record of things. My life, mostly. But lately I’ve either been to busy to post, or I’ve been using Twitter to record my daily doings. I’m not doing a very good job or recording much of anything here.

For a while now, I’ve been thinking about how nice it would be to have a database-driven website where you can record just about anything, and view stats and numbers associated with those records. For example, as a personal project to better learn PHP and MySQL, I’m creating a site where Ally can keep track of the books she’s read. I’m making it expandable enough that eventually, multiple users will be able to use the same site, see what other books people have added, and associate those books with their own accounts. By “associate” I mean that they will be able to add it to their wishlist, or they can mark it as “already read” and put it in their archive of completed books. They’ll be able to pull up all sorts of numbers and stats, see graphs of books completed within a date ranges, let the site choose their next book for them based on any number or criteria, and so-on.

But I’m digressing a bit. The point is that it will be a simple website, and it’ll only take a minute to enter a new book you’ve read, yet you’ll be able to get all sorts of rich information out of it. I wish I could recreate this site for about a million different things, not just book reading.

Baby steps. I don’t have the know-how to create an infinitely customizable site where users can create their own set of record keeping data based on whatever’s important to them. Yet. Someday. For now, I just need somewhere to put things so I can easily “remember” them later (i.e. use a search bar to find them).

I have therefore decided to use this blog as a device for just that: record stats about my life. And to start, I’m going to create a post that simply records the bands I saw last night at the Guster concert. Utilizing tags, I’ll be able to later pull up posts for all the concerts I’ve been to (starting from today).

If I had started this about 8 years ago, I wouldn’t have to guess how many times I’ve seen Guster. “Was last night 10 or 11? 12 maybe? ” No more wondering.

No Comments »
date Aug 13th 2010
author Mike
category Geek, Life
tags, , ,