KEK Wrapper Userfs Guide for HPSS 7.3  (as of 2010/08/20)

 

Table of Contents

 

Chapter 1. – KEK FD Wrapper 2006. 2

1.0  Overview of KEK FD Wrapper 2006 2

1.1. API Definition 2

1.1.1 KEK FD Wrapper 2006 - API Definition 2

Funtion:  OPEN.. 2

Funtion:  CREAT. 2

Funtion:  CLOSE. 2

Funtion:  READ.. 2

Funtion:  WRITE. 2

Funtion:  FSYNC. 3

1.1.2 KEK FD Wrapper 2006 are implemented by the following APIs 3

Funtion:  hpss_FD_open. 3

Funtion:  hpss_FD_creat 3

Funtion:  hpss_FD_read. 3

Funtion:  hpss_FD_write. 3

Funtion:  hpss_FD_close. 3

Funtion:  hpss_FD_fsync. 3

1.2 How to Compile and Link 3

1.3 Sample Program and Sample Makefile 4

1.3 Environment Variable – F_HPSS_BUFSIZE_MB=256 4

1.4 Prerequisite 4

Appendix A. –   KEK Wrapper 2000. 5

A.0 Overview. 5

A.1 KEK Wrapper 2000 – API Specification 5

Function: INIT() 6

Function: TERM() 6

Function: OPEN() 6

Function: CREAT() 6

Function: READ() 6

Function: WRITE() 7

Function: SYNC() 7

A.2 How to Compile and Link 7

A.3 Sample Programs and Sample Makefile 8

A.4  Update History 8

 

Chapter 1. – KEK FD Wrapper 2006

 

1.0  Overview of KEK FD Wrapper 2006 

 

The following gKEK FD Wrapper 2006h was provided as sample programs in year of 2006 for HPSS 6.2. It was modification of the KEK existing wrapper 2000 so it is quite similar with the KEK existing wrapper 2000. As results, your application change is minimum if your application was developed based on gKEK wrapper 2000h.

gKEK FD Wrapper 2006h was ported to HPSS 7.3 in year of 2010 so that it is available now.

 

 

For more details about these API, please refer to A.1 KEK Existing Wrapper 2000 – API Specification.

 

       KEK FD Wrapper 2006  (7)

1      close

2      creat

3      fsync

4      open

5      read

6      sync

7      write

 

1.1. API Definition

 

1.1.1 KEK FD Wrapper 2006 - API Definition

 

Funtion:  OPEN

int OPEN(const char *pathname, int flags, mode_t mode);

 

Funtion:  CREAT

int CREAT(const char *pathname, mode_t mode);

 

Funtion:  CLOSE

int CLOSE(int fd);

 

Funtion:  READ

ssize_t READ(int fd, void *buf, size_t count);

 

Funtion:  WRITE

ssize_t WRITE(int fd, const void *buf, size_t count);

 

Funtion:  FSYNC

int FSYNC(int fd);

 

 

1.1.2 KEK FD Wrapper 2006 are implemented by the following APIs

 

Above posix-like-naming APIs are implemented by the following HPSS APIs.

Please refere to HPSS Programmer's Reference for High Performance Storage System for other  APIs specification.

 

 

Funtionhpss_FD_open

int hpss_FD_open( const char    *pathname, int      flags, mode_t   mode) ;

 

Funtionhpss_FD_creat

int hpss_FD_creat( const char    *pathname, mode_t   mode);

 

Funtionhpss_FD_read

ssize_t hpss_FD_read( int     fd, const void   *buf, size_t  count);

 

Funtionhpss_FD_write

ssize_t hpss_FD_write( int     fd, const void   *buf, size_t  count);

 

Funtionhpss_FD_close

int hpss_FD_close( int     fd);

 

Funtionhpss_FD_fsync

int hpss_FD_fsync( int     fd);

 

 

1.2 How to Compile and Link

 

Headers to be include:

 

/opt/hpss/kek/wrapper.FD.2006/config.h

 

Flags to be specified:

 

CFLAGS = -I. -I/opt/hpss/include  -DLINUX -DHPSS51 -DLITTLEEND -D_BSD_SOURCE -D_THREAD_SAFE -DLINUX -D_LARGEFILE_SOURCE=1 -D_LARGEFILE64_SOURCE -D_POSIX_C_SOURCE=199309L -D_XOPEN_SOURCE 

LDFLAGS = -L/opt/hpss/lib

 

Libs and Objects to be linked:

 

LIBS =  -L/opt/hpss/lib -lhpss

/opt/hpss/kek/wrapper.FD.2006/dummies.o

/opt/hpss/kek/wrapper.FD.2006/FD_wrapper.o

 

1.3 Sample Program and Sample Makefile

 

You can find the sample program and sample Makefile in the following directory.

/opt/hpss/kek/wrapper.FD.2006

 

1.3 Environment Variable – F_HPSS_BUFSIZE_MB=256

 

The following envinroment variable is necessary to use new FD wrapper 2006. It is the number of bytes which specifies the buffer size whish wrapper uses internally in order to make HPSS performance better.

256 is recommended.

It is declared in /opt/profile.d/hpss.sh and /opt/hpss/profile.d/hpss.csh. Please read it before running FD wrapper 2006 application.

 

export F_HPSS_BUFSIZE_MB=256

 

1.4 Prerequisite

 

The following APIs are used to implement above wrappers. They can be also be used as follows in order to re-write your existing application to enable HPSS file I/O.

      

1      fclose hpss_Fclose

2      fflush hpss_Fflush

3      fopen  hpss_Fopen

4      fread  hpss_Fread

5      fseek  hpss_Fseek

6      ftell  hpss_Ftell

7      fwrite hpss_Fwrite

8      File   HPSS_FILE

 

If you want to update your existing Posix API program and enable HPSS file I/O, please include  the folloinwg #define statements in your header file.

 

#define       FCLOSE hpss_Fclose

#define       FFLUSH hpss_Fflush 

#define       FOPEN  hpss_Fopen

#define       FREAD  hpss_Fread 

#define       FSEEK  hpss_Fseek    

#define       FTELL  hpss_Ftell

#define       FWRITE hpss_Fwrite

#define       FILE   HPSS_FILE

 

Please refere to HPSS Programmer's Reference for High Performance Storage System for other  APIs specification.

 

Appendix A. –   KEK Wrapper 2000

 

A.0 Overview

These are APIs provided as sample programs in year of 2000 for HPSS 4.5. They were ported to HPSS 6.2 in year of 2006. They were ported to HPSS 7.3 in year of 2010 so that they are available now.

 

The macros/functions to HPSS POSIX-like wrappers for KEK applications have been available since year of 2000. These wrapper APIs are also available at KEK with HPSS 7.3 in year of 2010 and later.

The KEK existing wrapper 2000 API put more priority on performance rather than the compliance with the Posix standard. They buffered I/O so that they are fast on I/O of HPSS files.

 

A.1 KEK Wrapper 2000 – API Specification

 

       KEK Existing Wrapper 2000  (12)

1      close

2      creat

3      exit

4      fsync

5      init

6      init2

7      open

8      read

9      stat

10     sync

11     terminate

12     write

 

If you want to update your existing Posix API program and enable HPSS file I/O, please include  the folloinwg #define statement in your header file.

 

#define OPEN H_OPEN

#define CREAT H_CREAT

#define READ H_READ

#define WRITE H_WRITE

#define FSTAT H_FSTAT

#define FCNTL H_FCNTL

#define FSYNC H_FSYNC

#define INIT H_INIT

#define INIT2 H_INIT2

#define TERM H_TERM

 

Function: INIT()

Status: Modify

Details

INIT(void)            /* calls hpss_SetLoginContext(c); */

 

Initialize client API wrapper environment

 

 

Function: TERM()             

Status: Modify

Details

TERM(void)            /* calls hpss_PurgeLoginContext(c); */

 

Terminate client API wrapper environment

 

 

Function: OPEN()

Status: Modify

Details

int OPEN(

char          *pathname,

int           flags);

/* mode_t     mode); missing from macros – set to 644 */

 

This function only supports the following flags:

O_RDONLY

O_WRONLY

O_RDWR

O_APPEND

O_CREAT

O_EXCL

O_TRUNC

All other flags error as gNot Supportedh.  The COS value for a file uses the default COS for the HPSS system or subsystem.

 

 

Function: CREAT()           

Status: New

Details

int     CREAT(

char          *pathname,

mode_t mode);

 

The COS value for a file uses the default COS for the HPSS system or subsystem.

 

 

Function: READ()

Status: Modify

Details

       ssize_t       READ(

              int           hpss_fd,

              void          *buf,

              size_t count);

 

The function performs a buffered read of data from HPSS.

 

 

Function: WRITE()

Status: Modify

Details

       ssize_t       WRITE(

              int           hpss_fd,

              void          *buf,

              size_t count);

 

The function performs a buffered write of data to HPSS.

 

 

Function: SYNC()

Status: New

Details

int     SYNC(

int    hpss_fd);

 

The function will flush any remaining buffer output to an HPSS file.

 

A.2 How to Compile and Link 

 

Here is how to write Makefile.

 

CFLAGS = -I. -I/opt/hpss/include -I/opt/hpss/kek/wrapper.2000 -DHPSS_CONFIG_H  -DLINUX –DLITTLEEND -fPIC

WRAPPER       = /opt/hpss/kek/wrapper.2000/wrapper.o

CONFH         = /opt/hpss/kek/wrapper.2000/config.h

LDFLAGS = -L/opt/hpss/lib

LIBS =  -L/opt/hpss/lib -lhpss

 

Please use #ifdef HPSS_CONFIG_H and #ifdef HPSS62.

Please include /opt/hpss/kek/wrapper.2000/config.h in your source, which uses #ifdef HPSS_CONFIG_H.

Pleae use /opt/hpss/kek/wrapper.2000/wrapper.o with your object module, which uses #ifdef HPSS62.

Please link /opt/kek/hpss/lib/libhpss.so.

Please understand that KEK Wrapper 2000 requires #ifdef HPSS_CONFIG_H and #ifdef HPSS62 so please do not use conflict ones.

 

A.3 Sample Programs and Sample Makefile 

 

You can find the sample Makefile and headers at the following directory.

/opt/hpss/kek/wrapper.2000

 

A.4  Update History

2006/12/27:  Change libhpssposix.so directory because it was changed in HPSS 6.2.1.4

2009/02/24:  Mention HPSS Programmer's Reference for High Performance Storage System Release 6.2 July 2008 (Revision 2.0).

2009/02/27: Update title

2010/08/20: Changes for HPSS 7.3 upgrade. HPSS 6.2fs  libhpssposix.so withdrawn.

END OF FILE