Execute FTP commands in batch on the iSeries

Tim Brinker

08 Mar 2001

Rating: -3.90- (out of 5)

On the AS/400 create a flat physical file. This file will contain the User ID and Password to sign for the system, the FTP command will be

executed. This file will also contain the FTP commands (PUT, GET..).

Example Flat File Containing the FTP Commands

USERID PASSWORD

binary

put Library/File Library/File

quit

Replace the USERID and PASSWORD in the example file with a valid User ID and Password for the system the FTP will be executing to.

Now to execute the FTP file defined above in a batch job, you need to override the file created above to a file name of INPUT. Then all you

need to do is execute FTP with the System Name or IP Address.

Example CL Commands to Execute the FTP

OVRDBF FILE(INPUT) TOFILE(FILECREATEDABOVE)

FTP RMTSYS('100.100.1.1')

DLTOVR FILE(*ALL)

You can also automate FTP commands from the PC. To do this you need again to create a script (text) file on the PC that contains the FTP

commands you wish to execute.

Example PC Text File of FTP commands

open 100.100.1.1 {IP Address of the system you wish to connect to}

USERID {User ID for the System You Will Connect to}

PASSWORD {Password of the System you Will Connet to)

binary

get /qsys.lib/temp.lib/temp.file/qtemp.mbr e:qtemp

close

quit

Now to execute the FTP command to transfer files from the another system to your PC do the following.

Start the MS DOS Prompt and at the DOS prompt enter the following

ftp -s:FILENAME

replace FILENAME with the name that you saved the text file as.

Note: My FTP scripts have "binary" in them. You use binary when you wish to transfer AS/400 Save files between systems.

AS/400 FTP name formats

Lou Gerritse

03 Apr 2001

Rating: -3.22- (out of 5)

When FTPing an AS/400 from a non-AS/400 client the command 'quote site namefmt' can be used to display the name format currently used

by the FTP server. To switch between LIB and DIR naming conventions using an additional 1 or 0 appended to the above command switches

the name format to the preferred naming format.

Tech tips: Making file transfers redundant

by Glenn Robinson

Article Information

Article ID: 18919

Pub: iSeries NEWS UK

Dept: Feature Articles

Date: June 24, 2004

Printer Friendly

Other Articles By:

Glenn Robinson

Article Feedback:

• Feedback can be submitted only for articles that are less than six months old.

My last few articles have been concerned with using the IFS to access file systems on other servers and this month I am going to carry on in a

similar vein. On my travels to iSeries and AS/400 sites I still see many customers using Data Transfer and FTP to move data between their

OS/400 based systems and their Windows/*nix systems. In the majority of cases there is no real need to do this any longer because of the

additional functionality built in to OS/400 over the last few years.

Most customers using File Transfer just want to get a file or two from the system, save it to My Documents and then open it in Excel. I was

working with a customer a little while ago who said it took him around four days at the end of each month to run queries and then to

download some very large files; he'd being doing this for around a decade!

COPYING TO/FROM EXTERNALLY DESCRIBED FILES

In a previous Tech Tip I briefly discussed how to use CPYTOIMPF. I'll expand on the command here and touch on a few other commands

too. The easiest way to get files in to a format that Excel or OpenOffice can read is to use the Copy to Import File command (CPYTOIMPF).

This command takes a database file and converts it to csv format. This is an ASCII delimited file format that is understood by many

applications which separates each field with a comma, places double quotes around character fields and terminates each record with an End

Of Record character(s).

You will recall that the /QNTC file system will allow me to copy files to/from shares on other Windows servers, Samba servers and OS/400

systems running NetServer. Simply by using a command such as:

CPYTOIMPF FROMFILE(QIWS/QCUSTCDT)

TOSTMF('/qntc/fs001/public/glenn/customer.csv')

STMFCODPAG(*PCASCII)

RCDDLM(*CRLF)

I can get a csv format version of file QIWS/QCUSTCDT in to a share on the remote server FS001.

The important parameter shown above is STMFCODPAG. This determines what the code page of the file you are creating will be. If you

leave it as default and try to open the csv file in Excel it will just show up as a load of weird characters. By specifying *PCASCII you are

instructing OS/400 to convert the database file in to a csv format that Windows applications will understand.

The RCDDLM parameter lets you specify the characters that are used to denote the end of each record. By default this is set to *EOR which

adds hex 00 to the last field. If you are sending the file to a DOS/Windows machine, you need to specify *CRLF; for a *nix target system

specify *LF. In my example, I have given the target file a suffix of .csv. Although you could put anything at all, doing this will ensure that

Windows associates the file with the MS Excel application, ie when you double click the customer.csv file Windows knows to open it with

Excel.

Of course, you can also import delimited files into DB2 files too. CPYFRMIMPF allows you to take a file in csv format and copy the

contents in to an externally described file. The following command:

CPYFRMIMPF FROMSTMF('/qntc/fs001/public/glenn/customer.csv')

TOFILE(GLENN/CUST)

RCDDLM(*CRLF)

will take the contents of file customer.csv on server FS001 and adds it to file CUST in library GLENN. Note that the file GLENN/CUST

must already exist.

COPYING FLAT FILES

Sometimes you want to copy to or from a flat file. Let's say you've run the CPYSPLF command to get the contents of a spool file to a file;

this can be copied to the IFS or to a remote system using the CPYTOSTMF command:

CPYTOSTMF FROMMBR('/qsys.lib/glenn.lib/spoolfile.file/spoolfile.mbr')

TOSTMF('/qntc/fs001/public//glenn/customerReport.txt')

STMFCODPAG(*PCASCII)

As you can, see the format is similar to CPYTOIMPF but the first parameter is in IFS format rather than LIBRARY/OBJECT format. Again,

by specifying .txt Windows will know what program to open the file with -- .rtf and .doc also work. CPYFRMSTMF will copy a text file in

to a flat file in a library, again the format is similar to the above command.

SHARING THE FILES

The methods I have described in this and previous articles have shown how you can create ASCII versions of database files and copy them to

shares on remote servers or to the IFS on another iSeries system. One method I favour is to copy the files to a folder within the IFS of your

local iSeries rather than to a remote share. You can then share out the local folder using either a NetServer share or an NFS export. In both

cases you can specify that the share/export is read-only or read/write. The two benefits of using this method are that the data is held locally so

will be saved as part of your daily backup routine with the SAV command and that, by using a read-only share/export, the file cannot be

tampered with by the clients using the files. They can, of course, copy the file to their PC and amend it but they wouldn't be able to replace

the original file.

COPYING SAVE FILES

The CPY command allows to you move any IFS objects around. This is especially useful for moving Save Files between your iSeries

systems. Consider the command:

CPY OBJ('/qsys.lib/glenn.lib/glenn.file')

TODIR('/qfilesvr.400/london/qsys.lib/qgpl.lib')

This command takes a Save File called GLENN in library GLENN on my local system and copies it to library QGPL on remote iSeries

LONDON.

WHERE NOW?

Although using the IFS commands will replace many Data Transfer requests it will not replace all of them. One major benefit of replacing

Data Transfers with IFS commands is that you don't need an iSeries Access/Client Access license.

Most people don't realise that you only need an iSeries Access license for PC5250 and Data Transfer, all other iSeries Access and iSeries

Navigator functions are included as part of your OS/400 license. This includes ODBC, so you can quite happily use any application that

supports ODBC to get data from your OS/400 files and in to your Windows applications with no license required.

Going back to my previous comment about the customer who'd been doing the same file transfers every month for many years, I got to

thinking why he was still doing these transfers and came to this conclusion. Many iSeries and AS/400 shops are led to believe by Windows

technical staff and IT management that our system is an old legacy system so we feel that the way we did things ten years ago is still

appropriate today. This is not necessarily true; we have possibly the most robust, flexible, open and feature-rich computer system around but

we don't, in general, exploit it.

Hopefully my last few Tech Tips will have demonstrated how well iSeries integrates with other non-OS/400-based computer systems. My

advice is that you should question the way you move data around between your iSeries and other systems. In some instances your procedures

will still be valid but I believe that many of them would benefit from using some of the more up-to-date methods available within OS/400.

For direct FTP to a physical file:

1. The flat file needs to be fixed-length, matching character-for-character the layout of the physical file. The next 2 items describe things that

prevent this.

2. If the physical file has what are called packed fields, then not directly. It is possible to define what is called a logical file in DDS or SQL,

and that file can map the packed fields to what are called zoned fields. BTW, packed fields store each digit in half a byte, so each byte stands

for 2 digits.

3. If there are negative numbers, not usually - the sign is held in a half-byte of the last byte of the zoned field.

If none of those problems exist, this is very simple. There are a couple naming conventions to get used to, but it's just simple ftp at this point.

But if not, there is still hope - there is a command on the 400, CPYFRMIMPF (copy from import file) that can do the conversions, especially

well with delimited PC files. Actually, fixed-length PC files are harder with this command - they require another object on the 400 that

defines the layout of the data.

To use CPYFRMIMPF, the flat file has to get to what is called the IFS (Integrated File System) of the 400. This is a PC-like file system, and

you can either FTP the file to it, or use Windows networking to get there - map a network drive.

You can ftp the flat file to the IFS, then use a special 400 ftp server subcommand (rcmd) that can execute the CPYFRMIMPF command

remotely.

There's more to the details - the ftp manual covers a lot.

How do I do FTP from/to my AS/400?

I) Manual method - Good for one shotters or for testing and learning.

A) Starting ftp from my PC to the 400

From your PC do the following:

Start

Run

cmd

From the command interface window:

ftp your400name

Enter your userid

Enter your password

For a list of commands type in ? and hit enter

By default you will be in the library system. To change to the directory

system then enter: quote site namefmt 1

To change to the library system: quote site namefmt 0

get is used to get a file from the remote to the local.

put is used to put a file to the remote from the local.

Here is a sample used to run a command on the 400:

quote rcmd crtpf mylib/myfile rcdlen(80)

quit is used to exit.

When transmitting a save file to the 400, the save file should already

exist. Also you should use the bin command to transmit the file in

binary.

B) Starting ftp from my 400 to another 400

Very similar to starting ftp from a pc to the 400.

Differences include:

While you still use ? to prompt for commands the prompting appears

different.

C) Starting ftp from my 400 to a PC.

You'll need to ensure that the PC has a ftp server loaded. Windows comes

with one, but normally you have to load that separately. There are

others available.

II) Automated, or scripting, method

A) Scripted ftp from the pc to the 400.

First create a .BAT file. Let's call ours FTPSCRIPT.BAT. In it we'll

have one line:

ftp -n -s:ftpscript.txt>ftpout.txt

The -n says not to open a system until we say so.

The -s says to used a script file and this is the name of the script file.

The >ftpout.txt will pipe the output to a text file. You can check this

file for transmission errors.

Now create another text file called FTPSCRIPT.TXT. In this place:

open myas400

user myuserid mypassword

cd mylib

quote rcmd crtpf mylib/myfile rcdlen(80)

put myfile.txt myfile

quote rcmd runpost

quit

Now when you type ftpscript it will run the ftpscript.bat file, which

will use the ftpscript.txt file. You can use notepad or some other

utility to check the ftpout.txt file for the status of the transmission.

An important thing to remember is that the command we used, runpost, will

have to set up the library list. It will not use any library list

defined by your user profile's job description.

B) Scripted ftp from 400 to another server.

First create a simple CL program. Let's call ours FTPSCRIPT. In it we'll

have these lines:

PGM PARM(&RMTSYSTEM)

DCL VAR(&RMTSYSTEM) TYPE(*CHAR) LEN(200)

OVRDBF FILE(INPUT) TOFILE(MYLIB/FTPSCRIPT)

OVRDBF FILE(OUTPUT) TOFILE(MYLIB/FTPOUTPUT)

FTP RMTSYS(&RMTSYSTEM)

DLTOVR FILE(INPUT)

DLTOVR FILE(OUTPUT)

CALL PGM(ANALYZER) /* Analyze FTP output */

ENDPGM

The file MYLIB/FTPSCRIPT should be a one field file. The record length

is pretty flexible.

The file MYLIB/FTPOUTPUT should also be a one field file. Again, the

record length is pretty flexible.

In the file MYLIB/FTPSCRIPT put the following

<- There should be one blank line first. This cancels

prompting for the user name and password. The user

statement will handle that.

user myuserid mypassword

cd mylib

quote rcmd crtpf mylib/myfile rcdlen(80)

put myfile.txt myfile

quote rcmd runpost

quit

Now when you type CALL FTPSCRIPT it will run the CL program FTPSCRIPT,

which will use the FTPSCRIPT file. You can use DSPPFM or some other

utility to check the FTPOUTPUT file for the status of the transmission.

The program analyzer would be written to check out the contents of

FTPOUTPUT and process accordingly.

An important thing to remember is that the command we used, runpost, will

have to set up the library list. It will not use any library list

defined by your user profile's job description.

Q. When I ftp from an AS/400 to another, once logged in, the

default directory is QGPL and the file format is the

library format (0). I then have to type the NA 1 comand

to switch to the IFS format and issu a CD command to

position myself in the right directory. There is a home

directory value in the user profile but it does not seems to work.

A. At V4R5 use CHGFTPA AUTOSTART(*YES) NAMEFMT(*PATH) CURDIR(*HOMEDIR).

You can also set the list format to a Unix format (LISTFMT(*UNIX))

if you wish. After the change stop and restart the ftp servers.

Answer courtesy Guy Murphy via MIDRANGE-L 30 Nov 2001

Scott Klement is hosting an open source FTP API project at

http://klement.dstorm.net/ftpapi/ If you want to script an

FTP session, but don't relish parsing the log file to see

if your transfer was a success or not, take a look at this

code and contribute to it's further development!

For moving binary objects like Java class files

or save files (not source or other text),

be sure to use binary mode. If you forget

and transfer in ASCII mode, FTP will attmept

to translate (if necessary) from ASCII to EBCDIC

and vice-versa. Example:

From the PC side:

ftp youriSeriesorAS400Box

quote namefmt 1

cd /your/IFS/directory

bin

put yourProgram.class

From the OS/400 side:

ftp yourPCBox

namefmt 1

lcd /your/IFS/directory

cd your/PC/directory

bin

get yourProgram.class

Note that you can only connect to a PC from

the iSeries if the PC is running an FTP

server.

When using FTP to another AS/400, remember that

if the file doesn't exist, FTP will create a

"default" file for you. The attributes of this

default file will make it a simple flat file

with a record length equal to the first record

and having a single field.

If the file already exists, FTP will write the

new records over the existing record format, so

the field definitions will remain intact.

If you need to FTP a file to another AS/400 and

it doesn't exist at the target, either:

1) Create the file on the target before FTP, or

2) Use a save file

a) on source, CRTSAVF

b) on source, SAVOBJ to the save file

watch the TGTRLS!

c) on target, CRTSAVF

d) FTP the save file in binary mode

e) on target, RSTOBJ from the save file

f) on target, DLTF the save file

g) on source, DLTF the save file

I am having a problem where records are dropping from my file when ftp occurs through

a batch process using this same method. When sending interactively, the file arrives

For save files, you can use name format 1

to create the save file at the target machine:

nam 1

put /qsys.lib/buck.lib/mysavefile.savf /qsys.lib/buck.lib/yoursavf.savf

Copy IFS data from the iSeries to a Windows server

Jean-Marie Sauvageot

05 Feb 2004

Rating: -4.42- (out of 5) Hall of fame tip of the month winner

Want to copy IFS data from the iSeries to a Windows server using only one command? These steps will help you do just that.

1. Start NetServer on iSeries using iSeries Navigator (/Network/Servers/TCP/IP/iSeries NetServer) or OS/400 command STRTCPSVR

SERVER(*NETSVR)

2. Enter command MKDIR DIR('/qntc/Wservername'). Be aware to enter this command after each system IPL because all these QNTC links

will be cleaned up during IPL.

3. Use command WRKLNK OBJ('/qntc/Wservername') and then option 5=display to verify that you can see all shared folders on the

Window server. Be aware that user/password on iSeries and Windows sever must be the same.

4. Use command COPY OBJ(IFSpathname) TODIR('/QNTC/Wservername/foldername)

And that's it.

Note: If your Window server is not in the same IP subnet as your iSeries server, you'll have to indicate your WINS server address in the

iSeries NetServer properties in iSeries Navigator.

Ten things to remember when using the IFS Ron Turull 14 Apr 2003 Rating: -2.93- (out of 5)

Using the IFS can be a little tricky. Ron Turull gives you 10 things to keep in mind.

1. To access the IFS from a PC, you need the Client Access/400 V3R1M0 Windows Client or the Optimized for OS/2 Client. 2. Each individual file system (e.g., the OS/400 library system or the DLS file system) is a major subtree in the IFS, with only the root above

them. 3. In QSYS.LIB (the OS/400 library system), libraries are subdirectories, as are database files (because database files contain members).

Database file members are considered "objects" or "files" in the IFS, as are other OS/400 objects. 4. Each individual file system has it own naming conventions. For example, in the QSYS.LIB file system you must specify a period and the

object type after the object name (some examples: ".LIB" for libraries, ".FILE" for files, ".MBR" for database members, or ".OUTQ" for output queues). Also, some file systems, such as QOpenSys, are case-sensitive.

5. To access an object in any of the file systems via the IFS, you must specify a DOS-like path name to the object. For example:

/QSYS.LIB/MYLIB.LIB/MYFILE.FILE/MYFILE.MBR

specifies member MYFILE in file MYFILE in library MYLIB in the OS/400 library system. You can use the back-slash character () instead of the front-slash character (/) in path names.

6. Hard links cannot cross file system boundaries while symbolic links can. The QSYS.LIB and QDLS file systems support only a single hard link to an object (the one that is created automatically when the object is created). You cannot create symbolic links in either the QSYS.LIB and QDLS file systems. However, symbolic links pointing to objects in QSYS.LIB or QDLS can be created in other file systems.

7. Directory and file names are stored in Unicode format. This allows the IFS to automatically convert directory and file names from one code page to another. For example when a PC uses a different code page than the AS/400, directory and file names are converted automatically to the code page of the PC. Caveat: The code page you are using must contain the characters used in the names. Bottom line: Use international letters and symbols only (i.e., use only characters common to all code pages).

8. The IFS supports the single-character wildcard character '?' as well as the multiple-character wildcard character '*' . (DOS users should be familiar with this.) For example, this gives AS/400 users the ability to search for objects using a more complex search criteria than that provided by the standard AS/400 generic name facility (e.g., WRKOBJ TEST*). The IFS Work with Object Links command (WRKLNK) can be used to perform a more complex search (for example, WRKLNK '/QSYS.LIB/A?C?.* will find ABCD.LIB and ADCB.FILE, but not ABBCD.LIB). Unfortunately, wildcard characters are allowed only in the object name and not in any of the directories listed in the path.

9. For a list of APIs that can be used by AS/400 programs to access the IFS, see "Unix-type APIs" in the API Reference manual. 10. The IFS supports local sockets. This allows sockets programming on a single machine to implement single-box client/server

programming.