Blog Series: WoG, Cesky Krumlov; Day 2: Unix

Dr. Julian Catchen
University of Oregon
Full-time Unix Ninja

Topic: Unix

So in my prep blog on programming I went through some of the basics of command-line and ninja-ery. But really you cannot do Unix command-line justice unless you jump in and do it yourself with the UnixTutorial provided. Additionally, Julian's slides are up on that same page and there are great at illustrating comparisons between what we are used to (graphical user interface) and how that translates or what that looks like on the command-line (or in the terminal) and the slides are in pdf format, so download them and learn about the history of Unix and take the tutorial.

Highlights?

• Unix was originally developed by AT&T!
• Steve Jobs was fired from Apple, developed Nextstep, Apple went into the tubes, re-hired Steve Jobs--Steve Jobs promptly threw their operating system out and applied Nextstep which because OSX (it's Unix based).
• Google Android runs Linux
• Airplanes with personal movie systems run Linux
• Wireless internet routers run linux
• By the end of these two weeks he plans to make us Unix/Linux converts
• We googled 'unix commands' to obtain a cheat sheet of commands to help us if we forget, HEY there is also a cheat sheet I linked for you on the programming prep blog--go figure! :)
• By the end of this session you should know about the following, if not, go back to his slides and the Unix Tutorial

1. change directories
2. list files, list all files, list files that humans can read
3. move up and down commands on the command line
4. create directories
5. know what relative vs. absolute paths are
6. know how to figure out where you are
7. know three ways to 'get home' from anywhere on the computer system
8. know what tab and tab tab do and why they are totally cool
9. more, head, tail and cat
10. know how to unzip and de-tar files or do both at once
11. what is grep?
12. how to obtain line counts in a file

There is a slide called "Explore the file hierarchy" YOU CAN DO THIS WITHOUT WORKSHOP FILES! Explore your own computer file system! Huzzah!

My favorite quote of the evening:

"When you type 'cd ..' it's a like a worm hole that comes and sucks you up a directory, it's really cool" ~Dr. Julian Catchen, Workshop on Genomics, Cesky Krumlov 2013

Blog Series: WoG, Cesky Krumlov; Day 1: Amazon Cloud

The Amazon Cloud
Dr. Konrad Paszkiewicz
Director, Wellcome Trust Biomedical Bioinformatics Hub

Topic: Amazon Cloud

Pros

• No need to house/maintain servers
• No need to worry about backing up
• Only pay for what you use
• Upgrade are handled by Amazon
• You can expand and delete storage as you have need
• There are many many preconfigured virtual machines (VM) to pick from if you don't want to develop one on your own (QIIME, STACKS and Short read aligment all have their own VMs)

Cons

• You will pay for it; storage even when you are not using it, time using it etc and many researchers are resistant to the idea that they have to 'pay' for computing power and storage. And they are surprised by how much computational power and storage can cost. Researchers need to start being trained to think of about computing costs in their grant proposals--cost of programming and software, costs of hardware, costs of the people that will need to be hired to program and do analysis.
• Cost of an Amazon VM can run 0.20-3.00/hr; you also pay by the Gigabyte.
• Data transfer to your VM could be slow
• If the network is down you won't be able to access your VM
• Typical per month cost is $200-$400 depending on how much data you store and how extensively you use the machine. So you have to decide is it worth it? What are the costs and benefits of buying and maintaining your own server or computing system that's powerful enough to run analysis and store all your data versus using Amazon web services.

Read on if you are interested in the tutorial/exercise we went through today despite it most likely not being applicable to your situation unless you have access and/or are interested in implementing amazon cloud services.