BBCP

De Wiki de Calcul Québec
Aller à : Navigation, rechercher
Cette page est une traduction de la page BBCP et la traduction est complétée à 100 % et à jour.

Autres langues :anglais 100% • ‎français 100%

Sommaire

Description

BBCP is a file transfer tool, similar to scp and ftp, which is recommended to transfer files at high speed. Indeed, scp and ftp cannot reach the full speed available to most modern supercomputers. BBCP fixes some of the problems of the two aforementionned tools and is recommended to transfer large files toward or from our sites.

Usage

bbcp shares the syntax of scp and is therefore rather straightforward to use.

The basic syntax is as follow:

[name@server $] bbcp [options] file [user@]server:relative_path


This command will transfer a file from the local host to the specified server. Note that the path is relative to the $HOME directory on the remote host. You do not need to specify the user if it is the same as the one on the local host.

Like with scp, you will need to input your credentials to start the transfer.

If you want to download a file from a remote host to your local host, use the following syntax

[name@server $] bbcp [options] [user@]remote_host:path/to/file local/relative/path


For example, if you want to download the results.dat file, located on server.calculquebec.ca in your remote $HOME directory to your local $HOME directory, you could use the command

[nom@serveur $] bbcp user@server.calculquebec.ca:results.dat /home/user/


Options

Reaching higher speed (-s et -w)

To reach optimal transfer speed, you must use the -s et -w options.

The -s option sets the number of streams that are opened at the same time, and the -w option sets the size, in kilobytes, of the TCP window to use.

Best speed are typically obtained with 10 to 12 concurrent streams and a window size of 1200 kb. The options we recommend are therefore

[name@server $] bbcp -s 12 -w 1200K file user@server.calculquebec.ca:/path/to/target


Force existing files overwriting (-f)

The default behavior of BBCP is to stop transfer if the target file already exists on the target location. If you want to overwrite an existing file, use the -f option:

[name@server $] bbcp -f file user@server.calculquebec.ca:path/to/file


Progress report (-P number)

With the -P option, BBCP will show a progress report every number seconds. For example

[name@server $] bbcp -P 2 file user@server.calculquebec.ca:path/to/target


will show a progress report every two seconds. Note: Do not use rates smaller than 1.

Reversed connection (-z)

If you need to communicate with a server that is blocked by a firewall, you can attempt to reverse the connection with the -z option. This will request that the remote server initiates the connection with your local host. For example, with the command

[name@server $] bbcp -z file user@server.calculquebec.ca:/home/user/


the host server.calculquebec.ca will start the transfer, which will go around its own firewall.

Resume a partial transfer (-a and -k)

If a transfer fails, you can resume it from the point where it ended. To do so, two options are available: -a and -k.

The -a option continues the transfer of partially transfered files. To use -a, the -k option must have been used for the initial transfer.

With the -k option, BBCP keeps partially transfered files, allowing to resume the transfer at a later time. The default behavior without this option is to delete partially transfered files.

Therefore, if you use the first of the following commands, you will be able to use the second one if the first transfer fails.

 
 [nom@serveur $] bbcp -k file user@server.calculquebec.ca:/home/user/
 [nom@serveur $] bbcp -a -k file user@server.calculquebec.ca:/home/user/
 


Recursive copy (-R)

The -R option allows to recursively copy a directory and its content.

References

For more details, you may read the official BBCP documentation.


Briarée

The bbcp command is only available on tf-briaree.calculquebec.ca (here tf stands for "transfer files"). Please log on this precise host to do your file transfers.


Outils personnels
Espaces de noms

Variantes
Actions
Navigation
Ressources de Calcul Québec
Outils
Partager