Blog Series: WoG, Cesky Krumlov; Day 3: Unix, Part 2--Ninja-ery

Julian Catchen
University of Oregon
Unix Ninja

Topic: Unix Part 2

So unfortunately as with the quality control blog, I am or will be unable to give you files that we practiced on but Julian's slides are quite good. We learned about pipes and added on to our current knowledge of command line. We finished up the end of the first slide set... which included all the following commands:

• man [command you are confused about]: manual for commands that gives you all the options. Some man pages are more helpful than others but if you stare at it long enough it'll start to make sense. Often times there are examples of usage so pay attention to those.
• ls, gunzip, more, cat, head, tail, grep, wc...all that we learned yesterday--so don't forget it and today we ended up having to man some of those commands to learn the options.
• sort [filename], just as it sounds--depending on what kind of sort you want you have to specify an option (see man page for sort; ie. numeric, alphabetical)
• uniq [filename], as with sort, lots of options to tease things out of your file.
• cut [filename], we learned this before too--BUT I learned anew that it is only for column files today.
• tr "       " "," [filename]: translate command that changes all tabs to commas in the given filename.
• tab = Ctrl+V+tab or \t
• |, this is a 'pipe' read Julian's slides part 1

Example: What do you think I just did to this file?

cat batch_1.genotypes_1.loc | tr "    " "," | grep "^96053"

1. I grabbed the file = cat
2. I piped it to the command 'tr' specifying I wanted all tabs changed to commas
3. Then piped it to another command 'grep' (remember what grep does?)
4. With grep I specified I wanted to look for all entries where the beginning of the line (hat symbol) started with 96053

Julian's Slides Part 2

We didn't get through all these slides but they are great so have a gander...

What we did get through:

Blog Series: WoG, Cesky Krumlov; Day 2: Unix

Dr. Julian Catchen
University of Oregon
Full-time Unix Ninja

Topic: Unix

So in my prep blog on programming I went through some of the basics of command-line and ninja-ery. But really you cannot do Unix command-line justice unless you jump in and do it yourself with the UnixTutorial provided. Additionally, Julian's slides are up on that same page and there are great at illustrating comparisons between what we are used to (graphical user interface) and how that translates or what that looks like on the command-line (or in the terminal) and the slides are in pdf format, so download them and learn about the history of Unix and take the tutorial.

Highlights?

• Unix was originally developed by AT&T!
• Steve Jobs was fired from Apple, developed Nextstep, Apple went into the tubes, re-hired Steve Jobs--Steve Jobs promptly threw their operating system out and applied Nextstep which because OSX (it's Unix based).
• Google Android runs Linux
• Airplanes with personal movie systems run Linux
• Wireless internet routers run linux
• By the end of these two weeks he plans to make us Unix/Linux converts
• We googled 'unix commands' to obtain a cheat sheet of commands to help us if we forget, HEY there is also a cheat sheet I linked for you on the programming prep blog--go figure! :)
• By the end of this session you should know about the following, if not, go back to his slides and the Unix Tutorial

1. change directories
2. list files, list all files, list files that humans can read
3. move up and down commands on the command line
4. create directories
5. know what relative vs. absolute paths are
6. know how to figure out where you are
7. know three ways to 'get home' from anywhere on the computer system
8. know what tab and tab tab do and why they are totally cool
9. more, head, tail and cat
10. know how to unzip and de-tar files or do both at once
11. what is grep?
12. how to obtain line counts in a file

There is a slide called "Explore the file hierarchy" YOU CAN DO THIS WITHOUT WORKSHOP FILES! Explore your own computer file system! Huzzah!

My favorite quote of the evening:

"When you type 'cd ..' it's a like a worm hole that comes and sucks you up a directory, it's really cool" ~Dr. Julian Catchen, Workshop on Genomics, Cesky Krumlov 2013

Blog Series: Workshop on Genomics, Cesky Krumlov; Preparation--Programming

Dobrý večer! from Cesky Krumlov, Czech Republic! Ok that's enough Czech from me...

Section 8: Programming

So BioPerl and (I'm going to plug BioPython in here too--see below, PyCogent) are basically what they sound like, perl codes/scripts/modules (however is easier for you to think about it) and python code (Perl and Python being programming languages) geared toward applications in biological analysis.

Now my husband is a programmer turned bioinformatic programmer and his best advice is to jump right in and just keep using it. My main concern with that...aside from my inability to manifest a 36 hour day that would allow me to take on learning a computer langauge...is I don't use it every day in my job. This makes it difficult to say the least to retain all the commands in your head and even then--when in doubt 'Google!' The links above, the BioPerl one being suggested in the preparation materials for the workshop and the BioPython will give you an idea of the modules/programs that have been constructed using those languages and provide more links if you want to get further into the mire and meld programming with biological analysis. You will have to learn some basics of the language itself before jumping into the biological application of it just for functionality's sake.

Now that's all I'm going to say for the moment, but we'll come back to this...my husband who was trained as a computer scientist and is a programmer who just recently got thrown into the world of biology head first and is now programming (using Python) for bioinformatic analysis has a python tutorial and some advice to dispense to you all who aspire to move in that direction...but he has to write it up.

In the meantime, one thing you absolutely need to get comfortable with is the ominous black box called command-line. You simply have to learn how to navigate around your computer in command-line interface. The conference organizers have provided a helpful tutorial so we are going to go through that and I'll add as we go based on my own trial and error experiences.