Transferring Files to Your Own Server with Globus Online
I have been benefited from Globus online a lot as I have many files to search for mass spec everyday on computer cluster in my institution. In this post, I want to explore how to set up own server to send files back and forth from your desktop PC. This will be useful in general sending relatively large files from one place to the other.
First I am assuming you have a server computer you have full access to. In my case, I have a server at home running ubuntu 10.04.02. You need to discover the right distribution of globus-connect-multiuser program. You can see the list from here. The instruction for installation of globus connect multiuser is written here and please use it as guidance.
In my case, I couldn’t see the one for ubuntu, so I asked Globus team. They told me I should use “globus-repository-5.2-stable-lucid_0.0.3_all.deb”. Here is the steps to configure server for globus online multiuser.
1) Download package
>sudo curl -LOs http://www.globus.org/ftppub/gt5/5.2/stable/installers/repoglobus-repository-5.2-stable-lucid_0.0.3_all.deb
2) Intall Debian-based distribution>sudo dpkg -i globus-repository-5.2-stable-lucid_0.0.3_all.deb
3) Get update
>sudo aptitude update
4) Install globus-connct-multiuser> sudo aptitude-y install globus-connect-multiuser
5) Update configuration file. This file is present in /etc/ directory. To modify you need to have permission
>sudo vim /etc/globus-connect-multiuser.conf
There are quite few things you need to change in order to get it work. What I am going to show here is a minimum setting. For more detailed setting, please consult Globus online customer service.
First, you need to change following lines. Note: you need to remove % and s and semi colon (;) for the lines you need to configure.
L11 User = user_name_you_use_to_log_in_globus_online
L16 Password = your_password_for_globus_online
L22 Endpoint = same_as_User
L29 Name = server (whatever you want to call your server)
L103 Server = XXX.XXX.XXX.XXX (the server’s IP address)
L112 ServerBehindNAT = True
L193 server = XXX.XXX.XXX.XXX (the server’s IP address)
6) Run the installed program. This will take a few moments to be in effect
7) Check if essential ports are open (LISTEN). Type sudo lsof -i
Pay attention to the far right column. These are the status of ports currently used in your server. You can see port 7512 is open (LISTEN) for Myproxy, and gsiftp is also open (LISTEN). If you want to know the port number for gsiftp, you can look up in the configuration file.
This shows only the part of the file, but you can see port 2811 is used for gsiftp. Now ports are open for globus connect multiuser. But you need to make sure the ports are accessible (open) from remote computer. This site is easy to test whether certain ports on your server is actually open or not. You can simply type the IP address and port number (7512 and 2811). If it says ports are closed, you should check if portforwarding is correctly set on your router.
7) Go to globus online website. Log-in and go to Manage Data, and click manage endopoints. Here you are going to add your server.
Enter Endopoint Name : username#server
Choose Myproxy for Idneify Providers. Hostname should be the same IP addresses used above.
Leave Server DN empty.
Server Domain should be the same IP address used above.
Keep the default server port: 2811
Hit the Create Endpoint button. Then click the activate tag and hit activate now button. Now it will ask you to enter User name, Passphrase, Server DN and Credential Lifetime. Enter username and passphrase used to log-in your linux server. You can leave the Server DN empty and put some numbers (e.g. 24) for crediential lifetime. Then you will see an error message. Copy the text after MYPROXY_SERVER_DN=, go to Server and paste into the Server DN (no double quotations). Hit Save.
Try entering the linux user ID and passphrase, then activate again. This time it should be activated.
Now your server is activated and ready to transfer files. Go to Manage Data and click start transfer. Then enter your Endopoint for your server and click Go. Now you need to enter again your user ID and passphrase for linux server and credential time.
Once everything is successfully configured, you should see directory in the window. Now you can start transfer your files. Essentially this is to set up FTP server but you can transfer files with much faster speed.
Initially I had a problem transferring files. I saw directory structures on both sides, but when I initiated transfer, the transferred files had no contents. I could create and delete files & directories, but transferring files were unsuccessful. If you encounter a similar problem think about these possibility.
1) Linux firewall and/or your router firewall is blocking
2) Port forwarding is not set up correctly on your router
If firewall is blocking certain port, it may cause trouble sending files. Remember, globus-connect-multiuser uses port 50000-51000 by default. In my case, 2) was the problem. My router has port forwarding setting, but it separates specific port forwarding and port range forwarding. Once I fixed it, it works flawlessly.
If activation of server, connection to the server and port setting are done correctly, globus connect allows transferring files between your PC and server. If you look at port usage, you will see a new connection is established.