If you already know all the stuff which I write in the subtitle, please play around with the following packages rpart
, party
, partykit
.
1. Operation System (OS)
Operation System Wiki We are using different operating systems all days, now just focus on your computer:
- Windows (Our laptop, you need to pay)
- Unix and Unix like operation system
FAQs of OS
What’s your point to say this?
We need an app/software, which is called Bash, installed into your computer.
Why we need this?
- We can use ssh via Bash connect to our lunchbox server.
- We can use Bash easily run GUIDE.
- We can use other powerful GNU software. eg. grep, sed, awk
Why do we need to use grep, sed, awk? Can we just use excel, modified data by hand?
- Yes you can, IN SMALL DATA SET!
- One question Valid Number Leetcode, if you have 1,000,000 recorders.
- I may teach in the later semester.
Windows User
- Do I need to download/install Linux? Do I need to buy a Mac?
No! You don’t need to! Our server is free, you only need a software and connect to the server(lunchbox), then you can try everything! Also, Windows 10 has the bash. Don’t worry I will teach you some basic stuff.
- What should I do?
We only need Bash to manager our computer, also can connect to the server. That’s all, the following steps you can do to say whether you can get the bash in your computer
open command prompt;
type `bash`
If you have a problem with bash:
if you are using Windows 10:
see [Windows 10 Installation Guide](https://msdn.microsoft.com/en-us/commandline/wsl/about)
else if you are using Windows 7:
try install [UnxUtils](https://sourceforge.net/projects/unxutils/)
Mac User
- Do I need to download/install Linux? Do I need to buy a new computer and install Linux?
No! Linux is another type of MacOS, although there is some difference between MacOS and Linux, but you will hard to tell the difference. We are not computer science engineer. Don’t buy a new laptop and install Linux…
- What should I do?
- Mac already have the terminal.app, you can use spotlight to find it. Bash already running inside.
- [Optional] Mac also has another fancy terminal app call iterm2 I used it.
Other Linux
I think you should have the ability to figure out any questions by yourself. Please use StackOverflow wisely.
2. Bash 101
I hope you already have bash in your hand. Here is a great course
summary (Thanks to Professor Cecile Ane):
- directory structure, the root is
/
- relative versus absolute paths
- shortcuts:
.
,..
,~
.
current directory..
parent directory~
your home directory
- tab completion to get program and file names
- up/down arrows and
!
to repeat commands
whoami |
who am I? to get your username |
pwd |
print working directory. where am I? |
ls |
list. many options, e.g. -a (all) -l (long) -lrt (reverse-sorted by time) |
cd |
change directory |
mkdir |
make directory |
rm |
remove (forever). -f to force, -i to ask interactively, -r recursively |
rmdir |
remove (delete) directory, if empty |
mv |
move (and rename). can overwrite existing files, unless -i to ask |
cp |
copy. would also overwrite existing files |
diff |
difference |
wc |
word count: lines, words, characters. -l , -w , -c |
cat |
concatenate |
less |
because “less is more”. q to quit. |
sort |
-n for numerical sorting |
head |
first 10 lines. -n 3 for first 3 lines (etc.) |
tail |
last 10 lines. -n 3 for last 3 lines, -n +30 for line 30 and up |
uniq |
filters out repeated lines (consecutive). -c to get counts |
cut |
cut and return column(s). -d, to set the comma as field delimiter (tab otherwise), -f2 to get 2nd field (column) |
echo |
|
history |
shows the history of all previous commands, numbered |
! |
!76 to re-execute command number 76 in the history, !$ for last word or last command |
Things you can do
Question: I want to execute my guide anywhere, not by copy my guide to anywhere.
We all use hyperlink every day, we like to create a desktop icon in windows or drag the program into your mac Dock, so that we can easily execute the program. Why we can do it easily because we know where is the program. How to let our computer know where is
guide
or in the future, if you need install some software, how to let our computer know.
Bash has a variable call PATH, which specifying a set of directories where executable programs are located. So that when you type
guide
into terminal or command line, bash will search that directory, if there is a program callguide
, it will execute it, if not, they will saycommand not found
So that we need to tell our computer “next time you need to search an additional directory where guide is located”
The following steps help you do this (Only do it in the first time), and you can execute guide anywhere on your own computer after.
- Create the following directory on your computer
i). ~/local
ii). ~/local/bin
- Add
~/local/bin
into your environment PATH variable
mkdir -p ~/local/bin
echo 'export PATH=$PATH:~/local/bin' >> .bash_profile
source ~/.bash_profile
mkdir -p flag: no error if existing, make parent directories as needed
You can also use man mkdir
in bash to see the details
Windows or Linux
If you use bash in Windows command prompt, you need to download Linux version of guide.
Download or move your guide
program not zip file into ~/local/bin
cd ~/local/bin
wget http://www.stat.wisc.edu/%7Eloh/treeprogs/guide/linux/64bit/Intel/guide.gz
gunzip guide.gz
chmod a+x guide
guide
Mac
If you want a different version of guide, change the URL.
cd ~/local/bin
curl http://www.stat.wisc.edu/%7Eloh/treeprogs/guide/mac/absoft/guide.gz --output guide.gz
gunzip guide.gz
chmod a+x guide
guide
Try it!!
Go to any directory from bash via cd
in your computer, type guide
see what’s happening.
Server (Your friend’s computer)
- What is lunchbox?
lunchbox is our stat department server. Many of you should already have relative experience with server, just like AWS.
You can take it as your friend’s computer, and we can use your friend’s computer to run your own programs.
- Ok, why I need this?
You can definitely ignore it leave if you refuse to learn!
- When I need this?
When you need your friend’s computer. For me, I usually need to run some simulations thousands of times, some optimization jobs take weeks. So that I can throw everything into the server.
- But I need my computer to watch a youtube video.
- I don’t want to save lots of files into my own computer.
- I want to play computer game when my simulation is running.
- I just want to save my electronic bill…
- How to use lunchbox?
Change username to your own username! The password will not show anything, but you can keep typing, and use Enter
or return
to execute.
ssh username@lunchbox.stat.wisc.edu
lunchbox 101
Since it is your friend’s computer, in the beginning, it doesn’t have anything. Even don’t know what’s your homework.
Your friend: Ok, you can use my computer only for homework and computing, you can not play games or watch youtube, promise?
You: Sure, what do you have?
Your friend: I only have a bash terminal…haha, no chrome:P
You: …Ok… Do you haveguide
?
Your friend: YES!!! Type following in bash
~loh/public/ADDGUIDE
Your friend: You classmates also want to use my computer, so I want to save all your file into one place, OK?
You need to create a directory under /workspace, username is your username to login lunchbox
mkdir -p /workspace/username
cd /workspace/username
guide
Your friend: Let’s try R? DON’T JUST TYPE R!!!!! It will make my computer too slow.
srun R --no-save
You: Hope can I upload my homework?
Your friend: You can install SecureFX, but today let’s use another way to upload your file
Open a new Command Prompt or terminal. You new terminal is not connected to your friend’s computer yet, you can usels
to see everything on your computer not the server.
Your current terminal A already connected to your friend’s computer, then open terminal B.
- Open a new terminal B, or command prompt
- Upload your file via terminal B use command
scp
with flag-r
, which means directory, to lunchbox - Check your file from terminal A
More details about scp
use man scp
Windows
I hope you can find your homework easily by this way. Suppose your homework is in the directory
C:\Users\WinUserName\Desktop\stat479
bash
cd /mnt/c/Users/WinUserName/Desktop
ls # you should see stat479
scp -r stat479 username@lunchbox.stat.wisc.edu:/workspace/username
Mac
Drag your homework directory into your terminal, it will show where is your homework directory.
Suppose your homework is on your Desktop directory
~/Desktop/stat479
cd ~/Desktop
scp -r stat479 username@lunchbox.stat.wisc.edu:/workspace/username
lunchbox 102 submit job
You can follow the Professor Loh’s slide to create a submit file for guide_impute.r, I will just show a simple example to write “hello world” use R.
Still in terminal A.