Sep 04

FTP Guard4i gets new feature

One of our clients was interested in the FTP Guard4i product and wanted to secure their FTP environment from unauthorized access. We installed the product and set the security so that all FTP access would now be monitored and restricted. Unfortunately after a few minutes we had to turn off the security because the client had not understood just how much FTP activity was carried out on his system. This was a problem because they did see some attempts to access the system using FTP from unauthorized users yet they could not identify all the authorized users until they hit the site and were rejected by the security settings. At first we were just adding users as they showed up in the log after checking that they were in fact authorized, but that gave a number of issues because the FTP access used by the users was not built to recover when the request was rejected. So we eventually turned off the security and left it up to the normal object security to handle the issues until we came up with a solution.

This concerned us as we did not like the fact that FTP activity was going on and the client was unable to see just how bad the problem was. So we started to think about how we can show the problem exists while not affecting the existing processes. Eventually we made a change to the programs that would allow the security to circumvented while still logging exactly what and who used the FTP services. Now the client is able to see all activity and we can build the FTP security using the log information before implementing the fully secured environment.

FTP is very unsecure and should be turned off where possible, if you must have FTP services turned on we suggest you investigate the installation of a security and logging package such as our FTP Guard4i. Just understanding the level of FTP activity that is going on could help you determine just how exposed to data theft you are.

Chris…

May 16

Pagination now added to log viewer

One of the tasks we left out in the initial release of the PHP Interface of FTP Guard4i was the ability to set the page size when viewing the log entries. What we wanted to do was allow the number of log records displayed to be preset by the user, this would allow the retrieval of records to the page to be carried out a lot quicker than if all of the records were to be displayed. As part of this exercise we also decided to add a search button for data stored in certain columns of the database, this would allow you to say filter the records based on a certain object or on a certain user etc. and still provide a paged output.

The following is a sample screen where the sort parameter is the date and time column, because we provided the sort capability we do not need a search capability as well so no search box is displayed.

Paged Log View

Paged Log View

Here is a sample screen showing the sort column being the Object information and the search value was QSYS.

Paged View with Search

Paged View with Search

We are constantly looking at ways to add new features and functionality to the FTP Guard4i product, if you have any questions or would like to see a demo please let us know.

Chris…

May 06

FTP Guard4i is available for download

FTP Guard4i is now completed and available for download. We have placed the manuals online as well as the objects required to install the product. You will need to sign in as a member to download the objects and once installed you will need a key to allow the product to function. The PHP interface is available and requires the Easycom i5_toolkit functions to allow connectivity to the IBM i. We have not tested it with the Zend Free toolkit at this time and would need to make some additional changes due to the lack of support for some objects. If this is needed we can work with you to make those changes.

FTP Security is something we have been looking at for a long time, our initial requirement was highlighted because of the access to the source code for our products by the developers. We needed to give them access to the code to allow them to carry out their activities but we did not want them to be able to copy the code to other systems. The original product we created also provided an FTP Client so we could make the object transfer a lot easier than the FTP Client provided by the OS but this release only provides the security aspects required.

As part of the rewrite we have made a number of improvements in the methods we used to control the access particularly around the accept and reject IP addresses set for individual users. This allows you to set a range of IP addresses a user can connect to and from in the same manner as you can set the connection accept and reject addresses. We have also changed the logging to a Database file which allows us to add much more meaningful data about the activities carried out. While the clean up routines we have provided only allow the log to be cleared, using standard SQL against the file will provide a lot more granular entry removal.

FTP Security is an area most IBM i shops ignore because they believe the IBM i is naturally more secure than other platforms, that is not true and as we see more and more IBM i systems being linked to a wider audience we could see more intrusions being logged. FTP Guard4i also has a very comprehensive logging feature so you can now see who connects to your server and what they did while they were connected.

If you need more information about FTP Guard4i or would like to see a working demo please let us know using the demo request forms on the website.

Chris…

Apr 29

FTP Guard4i interfaces completed

We have finished the PHP interfaces for FTP Guard4i. The 5250 interfaces are going to remain pretty much the same due to the limitations set by UIM (80 columns does not fit all of the data) but we hope to eventually add some new screens once we work out what makes sense. The PHP interface uses the i5_toolkit functions to extract the data from the IBM i, this allows us to run the Apache server on a separate server which is better suited to running an Apache web server than the IBM i. We also have the same processes running under iAMP on the IBM i for testing and demonstration purposes if you wish to see a total IBM i implementation.

Here is a quick overview of the pages and the data that they show.

1. FTP Guard4i Status screen

FTP Guard4i Status

FTP Guard4i Status

The list of users who are connected to the FTP server is a new feature which is only available in the PHP interface for the initial release due to the limitations imposed by the UIM (5250) screens. We did some testing with multiple users to see exactly what users were logged in and when which provided some interesting results.
The FTP Server is the job which is listening on port 21, the SSHD Server is the job which is listening on port 22. The log writer is the job which processes all of the request events which have been created as a result of user connections, this data is stored independently so even if the log writer is not running the events will be recorded waiting for the log writer to be started. We have also listed the exit points which have been correctly registered for FTP Guard4i, if any of these exit points are inactive no FTP activity will be logged until they are reset and the FTP Server restarted.

2. FTP Guard4i Server Users

FTP Guard4i Server Users

FTP Guard4i Server Users

Access to the FTP Server can be limited in many ways, the above image shows all of the configuration aspects of the users who are allowed to access the FTP Server and what limitations if any are set for that user. You can directly control all aspects of the FTP Server activity for a particular user such as when the can connect and where from, you can determine if they can move around the library/directory structure or if they are jailed to a specific one. If a user tries to connect to a directory/library which they are not allowed they will automatically be connected to the default directory/library. The list format and Name format are set regardless of the actual FTP Server settings.

3. FTP Guard4i Client settings

FTP Guard4i Client Users

FTP Guard4i Client Users

The FTP Client which is available on the IBM i is generally open to all users, this can be a major security exposure as a user with sufficient access can link a FTP Server to the system (a PC running FileZilla Server or similar) and transfer objects off to the PC without any trace. With FTP Guard4i all FTP activity is logged and can be reviewed to see what users did when using the services. The controls provided can limit the target Server (IP Address) and what activities the user can carry out, including the directory/libraries which can be accessed.

4. FTP Guard4i Accept IP Address

FTP Guard4i Accept IP config

FTP Guard4i Accept IP list

You can set the addresses which the users can connect to the FTP Server from, this is in addition to the IP addresses which can be set in the User settings which can provide a very simple to manage access tool. The process will check for an accept address and reject address entry, if an entry matches a specific accept entry the connection will be allowed even if a reject entry matches which is less specific. The User settings are checked after the connection to verify the user can connect from the IP address after this check.

5. FTP Guard4i Reject IP List

FTP Guard4i Reject IP

FTP Guard4i Reject IP List

The above shows a single entry which states that everything is rejected which does not match an Accept entry.

6. FTP Guard4i Log

FTP Guard4i Log

FTP Guard4i Log view

The level of logging can determine what log entries are placed into the log, if it is set to log all entries you will see an entry for every request made to the server including the actual files and directories which have been involved. This can be very important for auditors who need to view all of the transactions a user carried out via the FTP Services on the IBM i.

7. FTP Guard4i Config.

FTP Guard4i config

FTP Guard4i Config

There are various control files which determine how FTP Guard4i runs, the PHP interface provides the ability to view or update those files.

As you can see FTP Guard4i is pretty much completed, all we need to do now is carry out some additional testing before we move to the release stage of the process. We will also provide a manual which will give more details on the various configuration parameters and how to manage the data which is logged.

If you are interested in FTP Guard4i and the security of the IBM i FTP Services let us know. We can provide online demos of the product and show how effective it is in locking down user FTP activities. Don’t wait until your data has been stolen, act today and give us a call.

Chris…

Apr 24

FTP Guard4i Log Viewer

As promised we have now developed the log viewer which shows the events which have been logged by the FTP processes. The log view has a number of columns each of which is sortable but the default sort is done by the Date and Time with the latest entry at the top. Here is sample view of the log on our test server.

FTP Guard4i log view

A sample of the events logged by FTP Guard4i.

A couple of interesting things came about while generating the log, you will see that we deleted a file ‘/home/CHRISH/??_???????`%??>?>????????’, one of the issues we all come across from time to time is where a file in the IFS has a strange name, deleting the file using the normal IFS commands is not possible as it will always return ‘File not found’ errors. Using FTP (actually we used FileZilla) you can see that we successfully deleted the file in question. The log also shows a ‘Send File’ operation, that was actually a get operation from the FTP client but the event gets logged as a ‘Server Send File’ operation..

The PHP interface is now pretty much complete but we need to do some more work on the UIM interface to align the data store with the actual output to the UIM Manager. Once that is finished and we have done some more testing FTP Guard4i will be available for download.

Chris…

Apr 23

FTP Guard 4i Take 2

We had been discussing the FTP Guard 4i with a prospect and they mentioned that they would like to be able to monitor the FTP Server and SFTP Server from the FTP interface. So we have added a couple of new features to the status screen that allow the user to administer the FTP Server and the SSHD server which is used for the SFTP connections.

Here is the new status screen

New FTP Guard 4i status screen

FTP Guard 4i take 2

One of the things we did notice when we added the new features and checked they functioned was the SFTP connection takes on the QSECOFR profile in the job and drops the original user profile. We need to take a look at this to see exactly what effect this has? We don’t allow the QSECOFR profile to connect via FTP or SFTP so the security we have set for the user as far as FTP is concerned still applied.

Let us know if you are interested in this kind of solution and what if any additional features you would like to see. The Log viewer is coming along and will be the subject of our next post.

Chris…

Apr 30

Object replication between systems


Object replication is standard in High Availability products and is provided using a variety of technologies that capture the change of an object and trigger a replication request. In our RAP product we use the Audit Journal to capture the changes and use our own replication tools to copy and restore the object to the target system.

This is all well and good for us as we have the in house skills to create the replication programs, what about those smaller shops who have no such skills? Usually they would have to rely on a process which will save an object to a save file, send it to the remote system and restore it on the remote system. FTP doesn’t work any better because you still have to save the object to the save file and FTP it before restoring it again from the save file, FTP of i/OS objects such as files,programs etc is not allowed..

There is a solution, we had recently looked at the Object Connect programs as the transport method, but it seemed like it required Opti Connect to be installed for it to work! We sent a request off to IBM asking for a change to allow normal TCP/IP devices to be used to carry the request as not many smaller shops could afford to run opti-connect! IBM came back and said this functionality was already provided in the OS in the form of the Enterprise Extenders.

A little research later and we now have objects being saved from the source system and restored to the target automatically, it even has library re-direction built in! Now there are some caveats with the process (there usually are) such as message queue content and data queue content is not carried across as part of the transfer, but thats no big deal for most shops.

Enterprise Extenders are touted as IBM’s replacement for AnyNet support, it allows APPN traffic to flow between systems over a TCP/IP network. To set this up between two of our systems we just had to configure the network attributes, create a controller on each system and the use commands to replicate the objects and it all worked. We created the link between our SHIELD2 and SHIELD3 systems using the following steps

First we changed the network attributes to support HIPR.
CHGNETA LCLCPNAME(SHIELD2) LCLLOCNAME(SHIELD2) ALWHPRTWR(*YES)
CHGNETA LCLCPNAME(SHIELD3) LCLLOCNAME(SHIELD3) ALWHPRTWR(*YES)

Then we created the controllers on each system
CRTCTLAPPC CTLD(APPCCTL) LINKTYPE(*HPRIP) RMTINTNETA(SHIELD3) RMTCPNAME(SHIELD3) USRDFN1(128) USRDFN2(128) USRDFN3(128)
CRTCTLAPPC CTLD(APPCCTL) LINKTYPE(*HPRIP) RMTINTNETA(SHIELD2) RMTCPNAME(SHIELD2) USRDFN1(128) USRDFN2(128) USRDFN3(128)

After varying on the controllers we tried out a replication request from Shield2 to Shield3.
SAVRSTOBJ OBJ(FILEb) LIB(RAPDTA2) RMTLOCNAME(APPN.SHIELD3) OBJTYPE(*FILE)

The object was successfully saved and restored to the target system without any problems!

Looking at the process it looks like IBM is using a save and restore process but in the manuals it states the object is not interrupted by the process, we take that to mean it is not locked but we have yet to prove that! The process was certainly slower than our replication process but in the end it works! This solution can be used on any V5R4 or upwards system and is probably a lot better than using a FTP process where you are saving and restoring the object around the FTP request. I think this will add another element to the HA on a Shoestring process which looks after the replication of journal data, you will of course have to build a method of detecting object changes but at least the save and restore to the remote system is handled for you.

We did try to find out where the Enterprise Extenders were installed (LPP?) but could not find any information, however you do have to specifically install the Object Connect LPP option for this to work at the very least.

Hope that adds another interesting option to those do it yourself DR projects out there who need to be able to add object replication to their journal replication set up!

Chris…

Jan 07

iLook shows promise

We finally received our download links and licenses for the LookSoftware iLook product just before Christmas. I have attached a couple of screen shots to show you the results obtained simply by running the product out of the box.

iLook is a technology preview for the main LookSoftware products, there are some screens which will not convert to a GUI even if they are IBM screens, if you try to display an unsupported screen a message is sent stating that the conversion process will not work. There are also a number of features which appear to be clumsy on first use, but reading the manual (yes we do have to sometimes) soon points the user in the right direction. If you find any issues you can always lodge them with support or add posts to the forums to get updates or answers.

Having played with the product we feel it is a good first pass at explaining the basic re-facing capabilities of the LookSoftware products. However it is just a preview and should be viewed as such the main LookSoftware products offer a lot more functionality. Luckily our screens from the FTP Client converted automatically and gave a nice new look to the product. iLook is free, you simply have to register your interest on the LookSoftware website and they will ship you download instructions and licenses by email.

If you have downloaded our FTP Client Version 6.1 this could be a nice easy (and free) way to add a GUI interface to it! I am sure our other products will have the same effect.

To get a copy of the software register your interest here http://www.looksoftware.com/ilook/ilook-application.aspx

The forum related to the product can be accessed once you register here http://www.looksoftware.com/register.aspx?returnurl=%2fuser-login.aspx

Here are the sample screens using the iLook product.

System Control Panel

iLook System Control Panel

Here is the FTP Client Version 6.1 sample screen using the iLook product.

FTP Client running under iLook

FTP Client running under iLook interface

I hope that is enough to wet your appetite, get going and register for the product now!

Chris…

Nov 17

How to remove invalid IFS names

We had a client call after he had managed to create an IFS directory which could not be deleted using the WRKLNK interface or the RMDIR command using our new FTP Client. We didn’t realize that the IBM OS based commands could not delete items on the IFS under certain circumstances, but we also found that those same commands would refuse to create the objects as well.

The client had created a directory called c:\ftpclnt in the root directory, trying to delete the item using option 4 from the WRKLNK command failed as did the RMDIR command with a message stating the object could not be found.

Here is a sample of the message sent

Additional Message Information

Message ID . . . . . . : CPFA0A9 Severity . . . . . . . : 40
Message type . . . . . : Diagnostic
Date sent . . . . . . : 11/17/09 Time sent . . . . . . : 18:09:12

Message . . . . : Object not found. Object is /c:/testdir.
Cause . . . . . : Object /c:/testdir, or a directory in the object path,
could not be found, or its type cannot be resolved by this function.
Recovery . . . : Correct the name or specify an object of the correct type.
To determine if the object exists, use the Work with Object Links (WRKLNK)
command. If the name exists, check the type of the object. If the name
contains symbolic link objects, ensure the path names they resolve to exist.
Retry the operation.

Bottom
Press Enter to continue.

F3=Exit F6=Print F9=Display message details
F10=Display messages in job log F12=Cancel F21=Select assistance level

Apparently this is because the OS commands interpret the \ character differently,as can be see above it created a path of /c:/testdir which reversed the ‘\’ and called it a directory yet didn’t create a subdirectory of /testdir under c:….
We looked at the code which allowed the object to be created and it was the Unix API mkdir(). We also tested the delete of the link using our product and it did as we expected and successfully deleted the object. The question was why did the IBM commands refuse to find the object.

We logged a call with IBM support and they sent us to the following information . Talking with support we explained that we had Googled the problem and didn’t find the document they had sent, they explained this was because the document is contained in a knowledge base that is not viewable by web robots.

So if you are having the same problems, hopefully Google will find this reference and save you some time.

I have also copied the content below just in case the link doesn’t work for some? The customer said his link was only able to be deleted by using Navigator or our program, so make sure you try a few options.

Document Title: Renaming or Removing Files from the Integrated File System That Have Names That Are Not Valid

Abstract

This document provides tips on how to delete documents and directories in the Integrated File System that were created with invalid names.

Document Description:
Certain applications may create invalid file names in the operating system Integrated File System. For example, a file name containing slashes or quotes is not valid and cannot be removed by the operating system file system code. When trying to rename or delete these files, the iSeries interprets the slashes and quotes as part of the file name and generates an error, CPFA0A9 Object not found, because this naming convention is not valid for the iSeries system.

Note: When this problem is encountered, the best way to remove or access these files is to use the same application that created them.

If the original application is not available or it is not able to remove these objects, there are some other options that can be used. These options include FTP, QShell, iSeries Navigator, IFS tools DLTIFSF, and CleanNames.

FTP:

FTP can be used to rename or delete files and directories with invalid names that contain only standard ANSI characters. For example, FTP can be used to delete or rename files with names containing a backslash (‘\’), but cannot handle names with embedded nulls or Unicode characters.

FTP Commands: REN (rename) and DEL (delete) for files, RMDIR (remove directory) for directories.

Note: Directories must be empty before they can be removed with the RMDIR command.

For example, a file was created in the root of the Integrated File System. The files is called \MYFILE.TXT and must be deleted. Below is a WRKLNK showing how this file looks in the operating system Integrated File System.
Work with Object Links

Directory . . . . : /

Type options, press Enter.
3=Copy 4=Remove 5=Next level 7=Rename 8=Display attributes
11=Change current directory …

Opt Object link Type Attribute Text
QPWXGRB DIR
QPWXGUM DIR
QSR DIR
QSYS.LIB DIR PROD System Library
QTCPTMM DIR
QVGN DIR
RONTEST DIR
\MYFILE.TXT STMF
Snyder DIR
More…
Parameters or command
===>
F3=Exit F4=Prompt F5=Refresh F9=Retrieve F12=Cancel F17=Position to
F22=Display entire field F23=More options

Take the following steps to remove the file:

Note: An FTP session may be started either from the iSeries Command Line (on the same iSeries system or on a different iSeries system) or it may be started from a PC DOS Command Prompt.
1 To start an FTP session to the IBM System i system, on the operating system command line type the following command:

FTP

Press the Enter key. You are prompted to sign on and type your password.
2 Once signed on, change the naming format from operating system to UNIX by issuing the NAMEFMT 1 command (quote site namefmt 1, and press the Enter key). The FTP session should respond with 250 Now using naming format “1″.
3 To change to the root of the Integrated File System, type the following:

CD /

Press the Enter key. Response from the iSeries family system should be 250 “/” is current directory.

If the file is located in a directory or a subdirectory rather than on the root of the Integrated File System, issue CD dirname, and press the Enter key. Response from the iSeries family system should be 250 “/dirname” is current directory.
4 Type the following:

DEL \MYFILE.TXT.

Press the Enter key. The response is 250 Deleted file /\MYFILE.TXT. This also works with the RENAME command.
Note: Remember that this is a UNIX format. Therefore, file names are case-sensitive.

The same steps may be used for removing directories with invalid names (such as \MYDIR). To do so, follow the steps above substituting the RMDIR command in place of the DEL command used in the example. If the directory which has the invalid character in the name contains other DIRs or STMFs, you will probably need to do a REN on the directory with the invalid character in the name and give it a valid name. At this point, you can use normal methods to delete the contents of the directory and remove it as you normally would.

QShell:

QShell can be used to remove some invalid file names, including those that contain a backslash as part of the name. To remove a name with a backslash, escape the character with an additional backslash or double quote the name.

Example: To remove “myfi\le” use ‘rm myfil\\le’ or ‘rm “myfi\le”‘

To use QSHELL to remove the files, do the following:
1 From an operating system command line, type the following:

STRQSH

Press the Enter key.
2 To change directory to the directory containing the invalid file name, type the following: CD mydir
3 To remove the file, type the following: RM “invalid file name”

Note: Double-quotes are required in the RM command.
4 Press F3 to end QSHELL.
iSeries Navigator:

The iSeries Navigator File Systems | Integrated File Systems option can be used to delete or rename files and directories with names that Windows considers invalid (and hence network drives can’t handle). This includes names like *.* or *dir and many special ANSI characters such as the trademark symbol (TM) and so on.

To use iSeries Navigator to remove or rename the files, do the following:
1 Open iSeries Navigator.
2 Expand My Connections.
3 Expand File Systems.
4 Expand Integrated File systems and locate the directory or file containing the invalid file name.
5 Right click on the directory or file name and chose the option to delete or rename the file.

IFSTOOLS DLTIFSF and RNMIFSF:

DLTIFSF and RNMIFSF can be used to delete or rename files or directories with names that contain a backslash.

Example (delete): CALL DLTIFSF ‘[filepath]‘

Example (rename): CALL RNMIFSF PARM(‘[filepathold]‘ ‘[filepathnew]‘

Information about downloading and installing IFSTOOLS is in Rochester Support Center knowledgebase document 19175649, Integrated File System Tools: DEL, DELTREE, CMDALL, CHGAUTALL, CHGOWNALL, QRYIFSLIB, DLTIFSF, RNMIFSF:

CleanNames:

CleanNames is a Java toolbox utility. It is the best option to use when cleaning up thousands of files or when file names include embedded nulls. CleanNames can clean up invalid directory and file names such as: “*.*”, “*name”, “\name”, name with embedded null, name with a Unicode character, and so on. It does not work for names of “.”, “..”, or names that include a forward slash (‘/’).

Command syntax:

CleanNames SystemName TargetDir [option]

SystemName – AS/400 system name as entered in DNS or local hosts table
or the AS/400 system IP address. The name “localhost”
is a valid name when run from the AS/400 jvm.
TargetDir – The directory from which to start work. This is the
directory to delete all files from or the directory
to search for other directories with invalid names.
[option] – The menu option to execute. If not supplied, the user
is prompted for one of following options:

1 Rename all files in the target directory to a valid name.
The names are qfrecov1, qfrecov2, etc…
2 Rename all directories in the target directory to a valid name.
The names are qdrecov1, qdrecov2, etc…”
3 List all objects in the target directory. Prompt to rename.

WRKLNK Option 4=Remove:

The WRKLNK command is not Unicode-enabled. It cannot work with files or directories that have ANSI or Unicode characters that do not exist in the operating system CCSID.

EDTF:

EDTF STMF(/) will list Stream Files and Directories on the root of the IFS. Normal IFS commands (5 to display, and so on) can be used to locate the Stream File with the invalid name.

Use opt 4 to delete file or opt 9 to delete a directory and its contents.

Chris…

Nov 13

FTP Client Version 6.1 available for free 30 day trial

As part of the revitalization of the FTP Security Manager which is now available for download, we took the opportunity to update the FTP client which has the same FTP functionality as the one shipped in the FTP Security Manager without the security controls.

This version provides a much smoother interface to the previous incarnation, it also recognizes the CCSID of the system allowing the conversion of EBCDIC to ASCII to be carried out correctly. We have added some new features which allow the user to switch between remote and local directory listings with the press of a key and removed some of the old screens making navigation of the product much easier. Options have been aligned with the options used when navigating the IFS using the WRKLNK command reducing some of the confusion when first using the product.

This version also brings a new feature which allows the user to edit a local file or display a remote file locally. This is carried out using the IBM supplied EDTF and DSPF commands. We have made many other changes in this release and continue to develop new ones such as SSL support which we hope to have in the next version.

If you want to check it out you can do so with a 30 day free trail which can be downloaded from our members section.

Those who have tried it so far love the ease of use it brings and the ability to remove the practice of moving objects via a PC to get them to the IBM ‘i’ because users prefer a better interface than the IBM FTP Client provides today.

Please take the time to download the product and try it out, we look forward to any comments you have..

Chris…