Using Work Servers / Parallel Servers

System Overview

The Data Analysis system offers total of eighteen work servers and six parallel servers.

NB) Only granted users can login to Parallel servers.

Work servers T114
Work servers T24
Parallel servers6

Main Features of Work Servers

  • SSH-login service
  • File access via General Parallel File System (GPFS) and HSM System (GHI)
  • Calculations with batch job scheduling using Platform LSF
  • Various softwares

Main Features of Parallel Servers

  • SSH-login service
  • File access via General Parallel File System (GPFS)
  • Calculations with batch job scheduling using Platform LSF
  • Various softwares (Mainly Gaussian)

NB) Parallel servers process calculation jobs themselves, unlike work servers which hand them to the batch servers. For this reason, when a job is being executed the response time of parallel servers to other requests (such as logins) might be unusually long.

Environment Requirements

SSH Version 2 must be used to access these servers remotely.

See below for more information on connections with SSH.

  • When using UNIX terminals such as Linux, simply use the ssh command.
  • If you are connecting from Windows, you can either install cygwin and execute ssh from its terminal or use a terminal emulator such as TeraTerm or PuTTY. If you use TeraTerm, choose UTF-8 TeraTerm Pro.

In addition, direct remote access is permitted on this system. See here for more details.

NB) Be sure to set the passphrase to the authentication keys for ssh. The keys which passphrase isn't set to are deleted by administrator. Also, do not place unnecessary keys.

Log In / Log Out

Work Servers

When logging in to work servers, use a delegate host name listed in the table below. If you use an abbreviation (e.g. login.cc) for the delegate host name it is sometimes not possible to connect. In that case, use the FQDN (e.g. login.cc.kek.jp).

Delegate Host NameHost NameOS
login.cc.kek.jpccw.cc.kek.jpcw01.cc.kek.jpScientific Linux 6.7
cw02.cc.kek.jpScientific Linux 6.7
cw03.cc.kek.jpScientific Linux 6.7
cw04.cc.kek.jpScientific Linux 6.7
cw05.cc.kek.jpScientific Linux 6.7
cw06.cc.kek.jpScientific Linux 6.7
cw07.cc.kek.jpScientific Linux 6.7
cw08.cc.kek.jpScientific Linux 6.7
cw09.cc.kek.jpScientific Linux 6.7
cw10.cc.kek.jpScientific Linux 6.7
cw11.cc.kek.jpScientific Linux 6.7
cw12.cc.kek.jpScientific Linux 6.7
cw13.cc.kek.jpScientific Linux 6.7
cw14.cc.kek.jpScientific Linux 6.7
-ccx.cc.kek.jpcx01.cc.kek.jpScientific Linux 6.7
cx02.cc.kek.jpScientific Linux 6.7
cx03.cc.kek.jpScientific Linux 6.7
cx04.cc.kek.jpScientific Linux 6.7
math.cc.kek.jpcx04.cc.kek.jpScientific Linux 6.7

[Example]

 $> ssh  ccw.cc.kek.jp
  • Once logged in with a delegate host name, work servers will be assigned by round robin. For this reason, you log in to a different host every time. Bear in mind that local settings on a particular node (e.g. crontab) are not supported for any of the work servers.
  • All user accounts and passwords are managed by LDAP. Thus, the same account name and password can be used to log in to any hosts.
    NB: Users of the old system can still use their account. However, their password has been reset.
  • In case a work server had a failure, it is taken out of the round robin selection. After a problem occurred, it takes time between the reconfiguration of the round robin and the moment it disappears from the selection. If you are not able to access a delegate host name, try to log in again. It is possible to log in to a different work server.

CPU time limitations on Work Servers

Work servers ( ccw.cc.kek.jp and ccx.cc.kek.jp ) limit on the CPU usage of a process.

Over 2hours ... Reduce the priority of the process
Over 1week  ... Kill the process

Memory limitations on Work Servers

The amount of memory usage for all general users is limited on each work server as mentioned below.
In case the memory utilization reaches the limit, a series of processes which use large amount of memory may be forced to terminate for system protection.
We appreciate your understandings.

[ccw.cc.kek.jp]

112GB

[ccx.cc.kek.jp]

240GB

[math.cc.kek.jp]

240GB

Parallel Servers

When logging in to parallel servers, normally you should use cp01.cc.kek.jp. The other parallel servers (cp02~cp06) are used by submitting jobs from cp01.

Changing Password

First Login

The first time you log in, the passwd command is automatically executed. Once you have changed your password, the session will terminate once, so you have to log in again. For password policies, please see the paragraph Password Rules. SSH ver2 is the required protocol to log in.

 WARNING: Your password has expired.
 You must change your password now and login again!
 Changing password for user a0019.
 Enter login(LDAP) password:                ← Input your current password
 New password:                              ← Input your new password
 Retype new password:                       ← Input your new password again
 LDAP password information changed for a0019
 passwd: all authentication tokens updated successfully. 

Changing Password (Any Time)

To change password, use passwd command;

[Example]

 $ passwd
 Changing password for user a0019.
 Enter login(LDAP) password:                ← Input your current password
 New password:                              ← Input your new password
 Retype new password:                       ← Input your new password again
 LDAP password information changed for a0019
 passwd: all authentication tokens updated successfully. 

NB) If you make a mistake when you input the current password, the passwd command will terminate without changing anything.

Password Rules

Avoid specifying a password based on a Dictionary Word and follow the rules below.

  • the password must have at least 9 characters,
  • it must include a least 1 non-alphanumeric symbole (ex: $ % *),
  • it must include a least 1 digit,
  • it must include at least 1 lowercase alphabet letter,
  • it must contain at least 5 different characters ("aa" is counted as 1, "ab" is counted as 2),
  • the last 4 passwords used cannot be re-use.

Changing Login Shell

To change your login shell, execute the command chsh with the option "-s" and the full path of your preferred shell as arguments. You can check the list of available shells by using the "-l" option. The change of shell will take effect at the next login.

NB) After chsh run, it can take up to 5 minutes.

[Example]

 $ chsh -l
 /bin/bash
 /bin/csh
 /bin/ksh
 /bin/sh
 /bin/tcsh
 /bin/zsh
 $ chsh -s /bin/tcsh
 Shell was changed.
 $

Environment Settings

Tcsh/csh

.cshrc
This file is invoked when a user logs in with ssh by csh or tcsh. Add or modify the file if neccesary. tcsh users should be aware that if you create a .tcshrc file in your home directory, the .cshrc file will be invalid.

NB) This file is located in the home directory.

NB) The originals of the files above are located under /sw/share/etc/skel on the work/parallel servers, you can refer to them if necessary.

Bash/sh

.bash_profile
This file is invoked when a user logs in with ssh on bash.
.bashrc
This file is invoked from .bash_profile. Add and modify this file if neccesary.

NB) These files are located in the home directory.

NB) The originals of the files above are located under /sw/share/etc/skel on the work / parallel servers, you can refer to them if necessary.

Tools Configuration Files

Each tool has environment configuration files located under /etc/profile.d.

The main tools added to this system are listed below along with their environment configuration files.

For csh/tcshFor ksh/bash/zsh
LSFlsf.cshlsf.sh
Textexlive.cshtexlive.sh
Environment Modulesmodules.cshmodules.sh
Allinea Forgeforge.cshforge.sh
MathematicaMathematica.cshMathematica.sh

NB) These files cannot be modified by normal users. Nevertheless, it is possible to overwrite the environment variables set in these files. If you change one of these variables to a wrong value, the tools will no longer work properly. Please be careful when modifying the value of an environment variable.

Environment Modules

In this system, "Environment Modules" is available to switch environment variables for using a software including several versions of compiler, MPI Library, Python, JAVA.

The Environment Modules allows users to set environment variables dynamically required for using each software. It provides easily way to changing environment variables when using different category or versions of each software.

NB) In the initial configuration, only OS standard version of each software is available. It is recommended to put "module load" command (refer to instructions below) in your shell configuration files which automatically invoked when logging in (.bashrc, .cshrc, etc.).

NB) When using each software in LSF jobs, you also need to load environment settings. Execute "module load" command to load environment settings in your shell configuration files which invoked when logging in or in your job scripts before using each software,

Available Modules

With the command below, all the available modules are listed.

 $> module avail↓

NB) "avail" can be abbreviated as "av".

[Example]

 $> module av↓
----------------------------- /opt/Modules/modulefiles -----------------------------
binutil/225 git/2171        mkl32/2016             openmpi/1.10.2-gcc485  OracleJDK/1.8      pgi32/2016
gcc/463    intel/2016    OpenJDK/1.6           openmpi/1.10.2-gcc530 pgi/16.3(default)     python/2.7
gcc/485    intel32/2016 OpenJDK/1.8           openmpi/1.10.2-intel     pgi/2016               python/3.5
gcc/530    mkl/2016      openmpi/1.10.2-gcc  openmpi/1.10.2-pgi       pgi32/16.3(default)


Loading an environment setting

With the command below, the environment setting ls loaded.

 $> module load <module name>↓

By executing "module load" command, environment variables such as PATH, etc. are set in the current executing shell environment.

[Example]

The syntax of loading the environment setting required for OpenMPI build with Intel compiler.

 $> module load openmpi/1.10.2-intel↓
$> which mpirun
/opt/openmpi/intel/openmpi-1.10.2/bin/mpirun

$> which icc
/opt/intel/compilers_and_libraries_2016.2.181/linux/bin/intel64/icc

NB) By loading the OpenMPI module, corresponding compiler module is loaded automatically.

Display environment variables that will be set

With the command below, you can confirm environment variables that will be set.

 $> module show <module name>↓

[Example]

$> module show openmpi/1.10.2-intel
-------------------------------------------------------------------
/opt/Modules/modulefiles/openmpi/1.10.2-intel:

prepend-path     PATH /opt/openmpi/intel/openmpi-1.10.2/bin
prepend-path     LD_LIBRARY_PATH /opt/openmpi/intel/openmpi-1.10.2/lib
prepend-path     MANPATH /opt/openmpi/intel/openmpi-1.10.2/share/man
-------------------------------------------------------------------

NB) You can check the environment variables of the compiler module that is automatically loaded when you load the OpenMPI module, by executing "modules show <compiler module name >" command.

The syntax of checking the environment setting of Intel compiler.

$> module show intel/2016

Display module names that are currently loaded

With the command below, you can confirm module names that currently loaded.

 $> module list↓

NB) "list" can be abbreviated as "li".

[Example]

$> module li
Currently Loaded Modulefiles:
  1) intel/2016             2) openmpi/1.10.2-intel

Additional loading modules and loading multiple modules

You can load additional modules by executing more than once "modules load" commands. You can also load Multiple modules at once by executing a "modules load" command.

[Example]

 $> module load openmpi/1.10.2-intel python/3.5↓
$> module li
Currently Loaded Modulefiles:
  1) intel/2016             3) python/3.5
  2) openmpi/1.10.2-intel

Unloading modules

With the command below, you can unload a module that currently loaded.

 $> module unload <module name that currently loaded>↓

[Example]

$> module unload openmpi/1.10.2-intel

Cleaning up modules that currently loaded

With the command below, you can unload all modules that currently loaded.

 $> module purge↓

NB) There are module command options "switch/swap" for swhiching modules, but these options somtimes does not work correctry, so we don't recommend to use these options. "purge" or "unload" options are recommened.

Languages

The standard language environment of work/parallel servers is UTF-8(en_US.UTF-8) which is an English environment. Other available codes include POSIX C, EUC(Japanese) and UTF-8. To change language, set the value of an environment variable LANG as follows. If not specified, the language environment will be set to UTF-8.

EnvironmentSetting
Englishen_US.UTF-8
POSIX CC
Japanese UTF-8ja_JP.UTF-8
Japanese EUCja_JP.eucJP
Shift JISja_JP.SJIS
  • Japanese EUC and SJIS are not officially supported but it is possible to use them.
  • However, file names with Shift JIS characters are not supported.

[Example]
With csh

 %> setenv LANG ja_JP.UTF-8↓

With bsh

 $> export LANG=ja_JP.UTF-8↓

When using Japanese UTF-8 or languages other than English, the terminal emulator setting on the client and the LANG variable on the server must match. In case of TeraTerm, the right correspondence between the two terminal settings is shown below.

TeraTerm SettingLANG Environment Variable
UTF-8ja_JP.UTF-8
Japanese EUCja_JP.eucJP

Using Japanese

Japanese input method is mostly dependent on terminal emulators used on client machines. For example, if you use TeraTerm on Windows, it is possible to use Japanese Input Methods available on Windows (MS-IME or ATOK...).

For more details, please refer to the manual of each method.

Emails

Receiving emails on work/parallel servers is not supported. Ountbound emails are not supported either, except for LSF job result notification emails.

Do not use the mail command.

Printing

List of Printers and Queues

Refer to Printer Information for the list of printers and printer queues.

Printing Output

Use the lpr command to print out.

 $> lpr -P Priter Queue Name  filename.ps

Example) Printing out on one side with printer DocuPrint C3250 (Queue:ccprt45ps1)in the Open Room in the Computing Research Center.

 $> lpr -P ccprt45ps1 test.ps

If the output file is not a ps file, you can convert it to ps format using the a2ps command.

 $> a2ps -o - filename | lpr -P ccprt45ps1

or

 $> a2ps -P ccprt45ps1  filename

Default printers are not configured, but if necessary you can set the environment variable PRINTER to the printer queue you usually use.

 %> setenv PRINTER ccprt45ps1      ← for csh
 $> export PRINTER=ccprt45ps1      ← for bsh 

If the command above is added in .cshrc or .profile file, after log in again, the command "lpr filename.ps" will print the file with the default printer ccprt45ps1.

NB) The a2ps command currently installed enables printing files written in UTF-8 character encoding. (Japanese only)

Example) tcsh is used in this example.

 %> setenv PRINTER ccprt45ps1
 %> setenv LANG ja_JP.UTF-8↓ 
 %> echo  'これは日本語のテストです。' > test.txt
 %> a2ps test.txt

In addition, the Perl version of this command, a2ps.pl has been installed in /usr/local/bin. It does not require to set up LANG, but it does require Japanese encoding to be EUC.

Example) tcsh is used in this example.

 %> setenv PRINTER ccprt45ps1
 %> /usr/local/bin/a2ps.pl test.txt | lpr↓ 

NB) Since /usr/local/bin has been included to the path, users can simply execute a2ps.pl without its absolute path.

Cancellation   

To view a job waiting atop of printer queue, or a job that is being printed;

 $> lpq↓

[Example]

 $> lpq↓
 ccprt45 is ready and printing
 Rank    Owner   Job     File(s)                         Total Size
 active  t0001   3677    test_c                          1024 bytes

To cancel a job waiting or printing;

 $> lprm↓

NB) You cannot cancel print jobs that have been already sent to the printer with this command. To cancel them, use the cancel button on the printer.

To cancel a waiting job in printer queue;

 $> lprm Job ID

[Example]

 $> lpq↓
 ccprt45 is ready and printing
 Rank    Owner   Job     File(s)                         Total Size
 active  t0001   3677    test_c                          1024 bytes
 $> lprm 3677

Check the print job ID with the lpq command, and then cancel it by using this number and the lprm command.

You can only cancel your own print jobs.

Other Software

It is also possible to use enscript (No Japanese) or paps+lpr to print files.

[Example with enscript]

 $> enscript  -P ccprt45ps1 --media=A4  test.txt
 [ 1 pages * 1 copy ] sent to ccprt45ps1

NB) In enscript, the default paper size is letter. It can be changed to A4 by specifying "DefaultMedia: A4" in the confiruation file (~/.enscriptrc).

[Example with paps+lpr]

 $> paps test.txt  | lpr -P ccprt45ps1

Important Point regarding Printers

If you wish to use any printer that is NOT listed in the Printer Information Page (including personal printers) from work servers, ask the administrator beforehand since network configuration changes might be required.

Online Manuals

Online manuals are available from command line on work/parallel servers.

man command

 $> man command name

[Example]

 $> man ls

info,pinfo command

 $> info command name
 $> pinfo command name

[Example]

 $> info ls
 $> pinfo ls

Additional Warnings

/tmp directory

/tmp shall be the location that makes basic system applications available to create temporary files.
Please refrain from creating large size files or number of files in the /tmp by yourself.
If the usage rate of the / tmp directory becomes high, large files are deleted.


Last-modified: 2019-04-10 (水) 18:14:34 (0m)