May 16

Pagination now added to log viewer

One of the tasks we left out in the initial release of the PHP Interface of FTP Guard4i was the ability to set the page size when viewing the log entries. What we wanted to do was allow the number of log records displayed to be preset by the user, this would allow the retrieval of records to the page to be carried out a lot quicker than if all of the records were to be displayed. As part of this exercise we also decided to add a search button for data stored in certain columns of the database, this would allow you to say filter the records based on a certain object or on a certain user etc. and still provide a paged output.

The following is a sample screen where the sort parameter is the date and time column, because we provided the sort capability we do not need a search capability as well so no search box is displayed.

Paged Log View

Paged Log View

Here is a sample screen showing the sort column being the Object information and the search value was QSYS.

Paged View with Search

Paged View with Search

We are constantly looking at ways to add new features and functionality to the FTP Guard4i product, if you have any questions or would like to see a demo please let us know.

Chris…

May 06

FTP Guard4i is available for download

FTP Guard4i is now completed and available for download. We have placed the manuals online as well as the objects required to install the product. You will need to sign in as a member to download the objects and once installed you will need a key to allow the product to function. The PHP interface is available and requires the Easycom i5_toolkit functions to allow connectivity to the IBM i. We have not tested it with the Zend Free toolkit at this time and would need to make some additional changes due to the lack of support for some objects. If this is needed we can work with you to make those changes.

FTP Security is something we have been looking at for a long time, our initial requirement was highlighted because of the access to the source code for our products by the developers. We needed to give them access to the code to allow them to carry out their activities but we did not want them to be able to copy the code to other systems. The original product we created also provided an FTP Client so we could make the object transfer a lot easier than the FTP Client provided by the OS but this release only provides the security aspects required.

As part of the rewrite we have made a number of improvements in the methods we used to control the access particularly around the accept and reject IP addresses set for individual users. This allows you to set a range of IP addresses a user can connect to and from in the same manner as you can set the connection accept and reject addresses. We have also changed the logging to a Database file which allows us to add much more meaningful data about the activities carried out. While the clean up routines we have provided only allow the log to be cleared, using standard SQL against the file will provide a lot more granular entry removal.

FTP Security is an area most IBM i shops ignore because they believe the IBM i is naturally more secure than other platforms, that is not true and as we see more and more IBM i systems being linked to a wider audience we could see more intrusions being logged. FTP Guard4i also has a very comprehensive logging feature so you can now see who connects to your server and what they did while they were connected.

If you need more information about FTP Guard4i or would like to see a working demo please let us know using the demo request forms on the website.

Chris…

Apr 29

FTP Guard4i interfaces completed

We have finished the PHP interfaces for FTP Guard4i. The 5250 interfaces are going to remain pretty much the same due to the limitations set by UIM (80 columns does not fit all of the data) but we hope to eventually add some new screens once we work out what makes sense. The PHP interface uses the i5_toolkit functions to extract the data from the IBM i, this allows us to run the Apache server on a separate server which is better suited to running an Apache web server than the IBM i. We also have the same processes running under iAMP on the IBM i for testing and demonstration purposes if you wish to see a total IBM i implementation.

Here is a quick overview of the pages and the data that they show.

1. FTP Guard4i Status screen

FTP Guard4i Status

FTP Guard4i Status

The list of users who are connected to the FTP server is a new feature which is only available in the PHP interface for the initial release due to the limitations imposed by the UIM (5250) screens. We did some testing with multiple users to see exactly what users were logged in and when which provided some interesting results.
The FTP Server is the job which is listening on port 21, the SSHD Server is the job which is listening on port 22. The log writer is the job which processes all of the request events which have been created as a result of user connections, this data is stored independently so even if the log writer is not running the events will be recorded waiting for the log writer to be started. We have also listed the exit points which have been correctly registered for FTP Guard4i, if any of these exit points are inactive no FTP activity will be logged until they are reset and the FTP Server restarted.

2. FTP Guard4i Server Users

FTP Guard4i Server Users

FTP Guard4i Server Users

Access to the FTP Server can be limited in many ways, the above image shows all of the configuration aspects of the users who are allowed to access the FTP Server and what limitations if any are set for that user. You can directly control all aspects of the FTP Server activity for a particular user such as when the can connect and where from, you can determine if they can move around the library/directory structure or if they are jailed to a specific one. If a user tries to connect to a directory/library which they are not allowed they will automatically be connected to the default directory/library. The list format and Name format are set regardless of the actual FTP Server settings.

3. FTP Guard4i Client settings

FTP Guard4i Client Users

FTP Guard4i Client Users

The FTP Client which is available on the IBM i is generally open to all users, this can be a major security exposure as a user with sufficient access can link a FTP Server to the system (a PC running FileZilla Server or similar) and transfer objects off to the PC without any trace. With FTP Guard4i all FTP activity is logged and can be reviewed to see what users did when using the services. The controls provided can limit the target Server (IP Address) and what activities the user can carry out, including the directory/libraries which can be accessed.

4. FTP Guard4i Accept IP Address

FTP Guard4i Accept IP config

FTP Guard4i Accept IP list

You can set the addresses which the users can connect to the FTP Server from, this is in addition to the IP addresses which can be set in the User settings which can provide a very simple to manage access tool. The process will check for an accept address and reject address entry, if an entry matches a specific accept entry the connection will be allowed even if a reject entry matches which is less specific. The User settings are checked after the connection to verify the user can connect from the IP address after this check.

5. FTP Guard4i Reject IP List

FTP Guard4i Reject IP

FTP Guard4i Reject IP List

The above shows a single entry which states that everything is rejected which does not match an Accept entry.

6. FTP Guard4i Log

FTP Guard4i Log

FTP Guard4i Log view

The level of logging can determine what log entries are placed into the log, if it is set to log all entries you will see an entry for every request made to the server including the actual files and directories which have been involved. This can be very important for auditors who need to view all of the transactions a user carried out via the FTP Services on the IBM i.

7. FTP Guard4i Config.

FTP Guard4i config

FTP Guard4i Config

There are various control files which determine how FTP Guard4i runs, the PHP interface provides the ability to view or update those files.

As you can see FTP Guard4i is pretty much completed, all we need to do now is carry out some additional testing before we move to the release stage of the process. We will also provide a manual which will give more details on the various configuration parameters and how to manage the data which is logged.

If you are interested in FTP Guard4i and the security of the IBM i FTP Services let us know. We can provide online demos of the product and show how effective it is in locking down user FTP activities. Don’t wait until your data has been stolen, act today and give us a call.

Chris…

Apr 24

FTP Guard4i Log Viewer

As promised we have now developed the log viewer which shows the events which have been logged by the FTP processes. The log view has a number of columns each of which is sortable but the default sort is done by the Date and Time with the latest entry at the top. Here is sample view of the log on our test server.

FTP Guard4i log view

A sample of the events logged by FTP Guard4i.

A couple of interesting things came about while generating the log, you will see that we deleted a file ‘/home/CHRISH/??_???????`%??>?>????????’, one of the issues we all come across from time to time is where a file in the IFS has a strange name, deleting the file using the normal IFS commands is not possible as it will always return ‘File not found’ errors. Using FTP (actually we used FileZilla) you can see that we successfully deleted the file in question. The log also shows a ‘Send File’ operation, that was actually a get operation from the FTP client but the event gets logged as a ‘Server Send File’ operation..

The PHP interface is now pretty much complete but we need to do some more work on the UIM interface to align the data store with the actual output to the UIM Manager. Once that is finished and we have done some more testing FTP Guard4i will be available for download.

Chris…

Apr 23

FTP Guard 4i Take 2

We had been discussing the FTP Guard 4i with a prospect and they mentioned that they would like to be able to monitor the FTP Server and SFTP Server from the FTP interface. So we have added a couple of new features to the status screen that allow the user to administer the FTP Server and the SSHD server which is used for the SFTP connections.

Here is the new status screen

New FTP Guard 4i status screen

FTP Guard 4i take 2

One of the things we did notice when we added the new features and checked they functioned was the SFTP connection takes on the QSECOFR profile in the job and drops the original user profile. We need to take a look at this to see exactly what effect this has? We don’t allow the QSECOFR profile to connect via FTP or SFTP so the security we have set for the user as far as FTP is concerned still applied.

Let us know if you are interested in this kind of solution and what if any additional features you would like to see. The Log viewer is coming along and will be the subject of our next post.

Chris…

Mar 10

ZendServer 5.6 Virtual Hosts problem

I am in the middle of major upgrade from Windows 7 to Windows 8 on my PC. Apart from a number of issues with IBM software and learning a totally new interface most of it has gone OK!. However one item which was really annoying was the migration of my test PHP environments.

I originally downloaded the latest version of Zend Server 6.0 for Windows which seemed to install OK on Windows 8 but gave me nightmares when trying to migrate my PHP websites over. The main problem is the lack of support from Aura for Zend Server 6.0 on Windows! I did correspond with Aura about when support would be available but at the time of writing we have no news. To overcome this we decided to download and install Zend Server 5.6 instead, the install went OK and the Easycom installer found the Zend install without any problems so we thought were were on a winner! Well that was until we tried to configure the VirtualHost containers.

In Zend Server 5.6, Zend appear to have changed the way that Virtual Hosts are configured, the way that we had been doing it for years seemed to work OK until we tried to connect to the websites from other devices. All we kept getting was the Zend Server test page, obviously something was wrong. When we looked at the configuration for the Apache server we found the following at the bottom of the config file.

Include “conf/zend.conf”
#ZEND-{090AFE0D4ABB5DFDCCB84641FE115680}
NameVirtualHost *:80

#ZEND-{9D752CB08EF466FC480FBA981CFA44A1}
Include “E:\Zend\ZendServer/etc/sites.d/zend-default-vhost-80.conf”
#ZEND-{9D752CB08EF466FC480FBA981CFA44A1}

#ZEND-{090AFE0D4ABB5DFDCCB84641FE115680}
#ZEND-{C5D84B491DE9533AB5043B62C3D41057}
Include “E:\Zend\ZendServer/etc/sites.d/globals-*.conf”
Include “E:\Zend\ZendServer/etc/sites.d/vhost_*.conf”
#ZEND-{C5D84B491DE9533AB5043B62C3D41057}

Now obviously the ‘#’ generally denotes a comment so we should be able to ignore those lines, but the other lines are what caused the issue. The zend.conf file contains the virtual host definitions for the GUI which is run on port 10081 so we could understand what was going on there. The next line seemed to bring in another virtual host configuration, and finally the “Include “E:\Zend\ZendServer/etc/sites.d/vhost_*.conf”" line brought in what should have been our defined virtual hosts.
Our configuration file called vhosts_shield.conf was added to the directory ‘E:\Zend\ZendServer/etc/sites.d’ so according to the config file we believed that it would be added to the Virtual Host configuration, but it was still refusing to server up the sites. After some review and a lot of searching the web (even the Zend documentation and forums) we could not find out where we should place the virtual host content for it to be served up correctly. We would always be sent the default Zend page by the server regardless of the url entered. At first we tried to amend the zend-default-vhost to point to our own directory structure (this is the same as we have always set up) but that kept stopping the server from starting. Then it dawned on us, the configuration file seemed to be forcing the Zend page and ignoring our own vhosts file even though we believed it was being loaded. So we decided to force the default VirtualHost to run against a a nondescript port and only have our virtual host file (sites.d/vhosts_shield.conf) running against port 80. That was it, simply changing the port configured in the httpd.conf file for the VirtualHhost made it work.

So if you have moved to Zend 5.6 or above and you are using VirtualHosts the above will work for you. Eventually we will find out where Zend likes to have the site content stored and how the config files should be set up, but for now we are OK. The Aura incompatibility with ZendServer 6.0 needs to be resolved as well, as soon as we know when Aura will support this configuration we will certainly add a new post.

Happy PHP’ing..

Chris..

Feb 12

Adding a Bar Graph for CPU Utilization to HA4i and DR4i

As part of the ongoing improvements to the PHP interfaces we have developed for HA4i and DR4i we decided to add a little extra to the dashboard display. The dashboard display is a quick view of the status for the replication processes, we used traffic lights as indicators to determine if the processes are running OK. Another post recently discussed using a gauge to display the overall CPU utilization for the system but we wanted to take it one step further, we wanted to be able to show just the CPU utilization for the HA4i/DR4i jobs.

We thought about how the data should be displayed and settled on a bar graph, the bars of the graph would represent the CPU utilization as a percentage of the available CPU and would be created for each job that was running in the HA4i/DR4i subsystem. This gave us a couple of challenges because we needed to determine just how many jobs should be running and then allow us to build a table which would be used to display the data. There are plenty of bar graph examples out there which show how to use CSS and HTML to display data, our only difference is that we would need to extract the data from the system and then build the bar graph based on what we were given.

The first program we needed to create was one which would retrieve the information about the jobs that are running that could be called from the Easycom program interface. We have already published a number of tests around this technology so we will just show you the code we added to allow the data to be extracted. To that end we extended the PHPTSTSRV service program with the following function.

typedef _Packed struct job_sts_info_x {
char Job_Name[10];
char Job_User[10];
char Job_Number[6];
int CPU_Util_Percent;
} job_sts_info_t;

int get_job_sts(int *num_jobs, job_sts_info_t dets[]) {
int i,rc = 0,j = 0; /* various ints */
int dta_offset = 0;
char msg_dta[255]; /* message data */
char Spc_Name[20] = "QUSLJOB QTEMP "; /* space name */
char Format_Name[8] = "JOBL0100"; /* Job Format */
char Q_Job_Name[26] = "*ALL HA4IUSER *ALL "; /* Job Name */
char Job_Type = '*'; /* Job Info type */
char *tmp;
char *List_Entry;
char *key_dta;
Qus_Generic_Header_0100_t *space; /* User Space Hdr Ptr */
Qus_JOBL0100_t *Hdr;
Qwc_JOBI1000_t JobDets;
EC_t Error_Code = {0}; /* Error Code struct */

Error_Code.EC.Bytes_Provided = _ERR_REC;
/* get usrspc pointers */
/* memcpy(Q_Job_Name,argv[1],26); */
QUSPTRUS(Spc_Name,
&space,
&Error_Code);
if(Error_Code.EC.Bytes_Available > 0) {
if(memcmp(Error_Code.EC.Exception_Id,"CPF9801",7) == 0) {
/* create the user space */
if(Crt_Usr_Spc(Spc_Name,_1MB) != 1) {
printf(" Create error %.7s\n",Error_Code.EC.Exception_Id);
exit(-1);
}
QUSPTRUS(Spc_Name,
&space,
&Error_Code);
if(Error_Code.EC.Bytes_Available > 0) {
printf("Pointer error %.7s\n",Error_Code.EC.Exception_Id);
exit(-1);
}
}
else {
printf("Some error %.7s\n",Error_Code.EC.Exception_Id);
exit(-1);
}
}
QUSLJOB(Spc_Name,
Format_Name,
Q_Job_Name,
"*ACTIVE ",
&Error_Code);
if(Error_Code.EC.Bytes_Available > 0) {
printf("QUSLJOB error %.7s\n",Error_Code.EC.Exception_Id);
exit(-1);
}
List_Entry = (char *)space;
List_Entry += space->Offset_List_Data;
*num_jobs = space->Number_List_Entries;
for(i = 0; i < space->Number_List_Entries; i++) {
Hdr = (Qus_JOBL0100_t *)List_Entry;
memcpy(dets[i].Job_Name,Hdr->Job_Name_Used,10);
memcpy(dets[i].Job_User,Hdr->User_Name_Used,10);
memcpy(dets[i].Job_Number,Hdr->Job_Number_Used,6);
QUSRJOBI(&JobDets,
sizeof(JobDets),
"JOBI1000",
"*INT ",
Hdr->Internal_Job_Id,
&Error_Code);
if(Error_Code.EC.Bytes_Available > 0) {
printf("QUSRJOBI error %.7s\n",Error_Code.EC.Exception_Id);
exit(-1);
}
dets[i].CPU_Util_Percent = JobDets.CPU_Used_Percent;
List_Entry += space->Size_Each_Entry;
}
return 1;
}

The program calls the QUSLJOB API to create a list of the jobs which are being run with a User profile of HA4IUSER (we would change the code to DR4IUSER for the DR4I product) and then use the QUSRJOBI API to get the CPU utilization for each of the jobs. We did consider using just the QUSLJOB API with keys to extract the CPU usage but the above program does everything we need just as effectively. As each job is found we are writing the relevant information to the structure which was passed in by the PHP program call.

The PHP side of things requires the i5_toolkit to call the program but you could just as easily (well maybe not as easily :-) ) use the XMLSERVICE to carry out the data extraction. We first created the page which would be used to display the bar chart, this in turn calls the functions required to connect to the IBMi and build the table to display the chart. Again we are only showing the code which is additional to the code we have already provided in past examples. First this is the page which will be requested to display the chart.

<?php
/*
Copyright © 2010, Shield Advanced Solutions Ltd
All rights reserved.

http://www.shieldadvanced.ca/

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Neither the name of the Shield Advanced Solutions, nor the names of its
contributors may be used to endorse or promote products
derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

*/
// start the session to allow session variables to be stored and addressed
session_start();
require_once("scripts/functions.php");
// load up the config data
if(!isset($_SESSION['server'])) {
load_config("scripts/config_1.conf");
}
$conn = 0;
$_SESSION['conn_type'] = 'non_encrypted';
if(!connect($conn)) {
if(isset($_SESSION['Err_Msg'])) {
echo($_SESSION['Err_Msg']);
$_SESSION['Err_Msg'] = "";
}
echo("Failed to connect");
}
?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Untitled Document</title>
<style>
td.value {
background-image: url(img/gl.gif);
background-repeat: repeat-x;
background-position: left top;
border-left: 1px solid #e5e5e5;
border-right: 1px solid #e5e5e5;
padding:0;
border-bottom: none;
background-color:transparent;
}

td {
padding: 4px 6px;
border-bottom:1px solid #e5e5e5;
border-left:1px solid #e5e5e5;
background-color:#fff;
}

body {
font-family: Verdana, Arial, Helvetica, sans-serif;
font-size: 80%;
}

td.value img {
vertical-align: middle;
margin: 5px 5px 5px 0;
}

th {
text-align: left;
vertical-align:top;
}

td.last {
border-bottom:1px solid #e5e5e5;
}

td.first {
border-top:1px solid #e5e5e5;
}

table {
background-image:url(img/bf.png);
background-repeat:repeat-x;
background-position:left top;
width: 33em;
}

caption {
font-size:90%;
font-style:italic;
}
</style>
</head>

<body>
<?php get_job_sts($conn,"*NET"); ?>
</body>
</html>

The above code shows the STYLE element we used to form the bar chart, normally we would encompass this within a CSS file and include that file, but in this case as it is just for demonstrating the technology we decided to leave it in the page header. the initial code of the page starts up the session, includes the functions code, loads the config data which is used to make the connection to the IBMi and then connects to the IBMi. Once that is done the function which is contained in the functions.php called get_job_sts is called. Here is the code for that function.


/*
* function to display bar chart for active jobs and CPU Usage
* @parms
* the connection to use
*/

function get_job_sts(&$conn,$systyp) {
// get the number of jobs and the data to build the bars for

$desc = array(
array("Name" => 'NumEnt', "io" => I5_INOUT, "type" => I5_TYPE_INT),
array("DSName" =>"jobdet", "count" => 30, "DSParm" => array(
array("Name" => "jobname", "io" => I5_OUT, "type" => I5_TYPE_CHAR, "length" => "10"),
array("Name" => "jobuser", "io" => I5_OUT, "type" => I5_TYPE_CHAR, "length" => "10"),
array("Name" => "jobnumber", "io" => I5_OUT, "type" => I5_TYPE_CHAR, "length" => "6"),
array("Name" => "cpu", "io" => I5_OUT, "type" => I5_TYPE_INT))));
// prepare for the program call
$prog = i5_program_prepare("PHPTSTSRV(get_job_sts)", $desc, $conn);
if ($prog == FALSE) {
$errorTab = i5_error ();
echo "Program prepare failed <br>\n";
var_dump ( $errorTab );
die ();
}
// set up the input output parameters
$parameter = array("NumEnt" => 0);
$parmOut = array("NumEnt" => "nbr", "jobdet" => "jobdets");
$ret = i5_program_call($prog, $parameter, $parmOut);
if (!$ret) {
throw_error("i5_program_call");
exit();
}
echo("<table cellspacing='0' cellpadding='0' summary='CPU Utilization for HA4i Jobs'>");
echo("<caption align=top>The current CPU Utilization for each HA4i Job on the " .$systyp ." System</caption>");
echo("<tr><th scope='col'>Job Name</th><th scope='col'>% CPU Unitlization</th></tr>");
for($i = 0; $i < $nbr; $i++) {
$cpu = $jobdets[$i]['cpu']/10;
if($i == 0) {
echo("<tr><td class='first' width='20px'>" .$jobdets[$i]['jobname'] ."</td><td class='value first'><img src='img/util.png' alt='' width='" .$cpu/2 ."%' height='16' />" .$cpu ."%</td></tr>");
}
elseif($i == ($nbr -1)) {
echo("<tr><td class='last' width='20px'>" .$jobdets[$i]['jobname'] ."</td><td class='value last'><img src='img/util.png' alt='' width='" .$cpu/2 ."%' height='16' />" .$cpu ."%</td></tr>");
}
else {
echo("<tr><td width='20px'>" .$jobdets[$i]['jobname'] ."</td><td class='value'><img src='img/util.png' alt='' width='" .$cpu/2 ."%' height='16' />" .$cpu ."%</td></tr>");
}
}
echo("</table>");
return 1;
}

The program call is prepared with a maximum of 30 job info structures, we would normally look to define this before the call and set the actual number of jobs to extract but for this instance we simply decided that 30 structures would be more than enough. After the program is called and the data returned we then build the table structure that will be used to display the data. We originally allowed the bar to take up all of the table width but after testing on our system which has uncapped CPU found that we would sometimes get over 100% CPU utilization. We still show the actual utilization but decided to halve the bar width which gave us a better display.

HA4i is running on our system in test so the CPU utilization is pretty infrequent even when we run a saturation test, but the image capture below will give you an idea of what the above code produces in our test environment.

CPU_Bar_Chart

CPU Ultization Bar Chart HA4i

Now we just need to include the relevant code into the HA4i/DR4i PHP interfaces and we will be able to provide more data via the dashboard which should help with managing the replication environment. You can see the original bar chart on which this example was produced here

Happy PHP’ing.

Chris…

Feb 07

Slow Response with i5_pconnect().


While experimenting with the latest version of our DR4i php interface we came across a slight issue with the i5_connection routines. The problem only appeared after we moved the code from our PC testing environment to the iAMP install so we thought it was simply a slow down as we moved from the PC to the IBMi, unfortunately this is only part of the problem. As soon as we found the issue we contacted Aura and asked them for support, they came back asking about how the problem was manifesting itself as they have not seen it elsewhere and were not sure what could be causing the problem.

We asked Aura about the code and what could have changed to cause the significant slow down, they said that nothing had changed and because they were not able to recreate the same issue in their network they could not understand why we were. After some further discussion and discovery they let us know that they had moved away from the gethostbyname() API to the getaddrinfo() API in preparation for IPV6 support. getaddrinfo() is the API which should be used in place of gethostbyname() API where IPV6 support is required.

We scoured the internet and found a number of entries which discussed the slowdown of lookups when getaddrinfo() was used. It was obviously a problem and we needed to understand how this was playing a part in our environment but not in Aura’s. So our first action was to write a test program which would take a host name and try to resolve that using the getaddrinfo() API. Here is the code we started off with.


#include
#include
#include
#include
#include
#include /* CEE date functions */

#ifndef NI_MAXHOST
#define NI_MAXHOST 1025
#endif

int main(int argc,char **argv) {
int error;
int junkl; /* Int holder */
double secs; /* Secs holder */
char Time_Stamp[18]; /* Time Stamp holder */
char hostname[NI_MAXHOST] = ""; /* Host name returned */
unsigned char junk2[23]; /* Junk char string */
struct addrinfo *result;
struct addrinfo *res;

CEELOCT(&junkl, &secs,junk2,NULL);
CEEDATM(&secs,"YYYYMMDDHHMISS999",Time_Stamp,NULL);
printf("Start = %s\n",Time_Stamp);
error = getaddrinfo(argv[1], NULL, NULL, &result);
/* time now */
CEELOCT(&junkl, &secs,junk2,NULL);
CEEDATM(&secs,"YYYYMMDDHHMISS999",Time_Stamp,NULL);
printf("After getaddrinfo = %s\n",Time_Stamp);
if(error != 0) {
fprintf(stderr, "error in getaddrinfo: %s\n", gai_strerror(error));
exit(EXIT_FAILURE);
}
/* loop over all returned results and do inverse lookup */
/* loop over all returned results and do inverse lookup */
for(res = result; res != NULL; res = res->ai_next) {
error = getnameinfo(res->ai_addr,
res->ai_addrlen,
hostname,
NI_MAXHOST,
NULL,
0,
0);
if(error != 0) {
fprintf(stderr, "error in getnameinfo: %s\n", gai_strerror(error));
}
if(*hostname != '\0')
printf("hostname: %s\n", hostname);
CEELOCT(&junkl, &secs,junk2,NULL);
CEEDATM(&secs,"YYYYMMDDHHMISS999",Time_Stamp,NULL);
printf("After getnameinfo = %s\n",Time_Stamp);
}
freeaddrinfo(result);
return 0;
}

When we ran this test program against our network with a simple hostname which is defined in our HOST file here is a sample of the output.

Start = 20130205095444797
After getaddrinfo = 20130205095453000
hostname: SHIELD3.SHIELD.LOCAL
After getnameinfo = 20130205095453000
Press ENTER to end terminal session.

This showed an 8 second response time for the getaddrinfo() API! Obviously this would not be acceptable as it would be used each time a connection was made. This was an issue because we do not have a DNS to resolve our local names and instead rely on the HOST table entries, our default search is set to *LOCAL so we would have expected getaddrinfo() to look up the address in the HOST table first and it would have been resolved. But due to the way the API has been coded it was always going out to the DNS server asking for an IPV6 address before looking for the IPV4 address in the HOST table.

We then looked at the documentation a lot closer and after some experimentation found that if we removed the Domain Information from the TCP/IP setup (option 12 on the CFGTCP menu) we could get the request for a server name back to immediate responses, but as soon as we added Domain information such as ‘shield3.shield.local’ the response time would instantly creep back up to over 8 seconds. Again not acceptable as the environment we needed the fix for is using NamedVirtualHosting which would always pass in a FQDN.

This is when we raised a PMR with IBM and supplied them with all of the data we had been using and asked for support. They came back with a link to a document which described the problem exactly and it was only affecting i/OS from V6R1 onwards. Because from V6R1 onwards IBM had implemented the getaddrinfo() API to do IPV6 lookups first it would always go out to the DNS for a name resolution even if an IPV4 address could be resolved from the HOST file! It would only drop back to a IPV4 lookup after the IPV6 lookup had failed!

The answer in the end was very simple, we just had to code up the AI_ADDRCONFIG flag in the getaddrinfo() request and it would only do an IPV6 lookup if more than 1 IPV6 address had been configured (::1 is not considered a configured IPV6 address). Now we see immediate responses from the API and everything works as it should even with the Domain Information configured.

If you are seeing a dramatic slowdown in your TCP/IP connection after migrating to V6R1 and you or your aplication vendor are using the getaddrinfo() API you may want to consider the above. Easycom connection routines are affected at the moment but a fix is being developed to resolve the issue.

Chris…

Dec 18

New features added to HA4i

A couple of new features have been added to the HA4i product as a result of customer requests. Auditing is one area where HA4i has always been well supported but as customers get used to the product they find areas they would like to have some adjustments. The object auditing process was one such area, the client was happy that the results of the audits were correct but asked if we could selectively determine which attributes of an object are to be audited as they have some results which while they are correct are not important to them.

The existing process was a good place to start so we decided to use this as the base but while were making changes improve the audit to bring in more attributes to be checked. We determined a totally new set of programs would be required which would include new commands and interfaces, this would allow the existing audit process to remain intact where clients have already programmed them into their schedulers and programs. The new audits would run by retrieving the list of parameters to be checked from a control file and only compare configured parameters. The results have been tested by the client and he has given us the nod to say this meets with his approval. We also added new recovery features which allow out of sync objects to be fully repaired more effectively.

Another client approached us with a totally different problem, they were having problems with errors being logged from the journal apply process due to developers saving and restoring journaled objects from the production environment into test libraries on the production system. This caused a problem because the objects are automatically journaled to the production journal when they are restored, so when the apply process finds the entry in the remote journal it tries to create the object on the target system and fails because the library does not exist. To overcome this we amended the code which supports the re-direction technology for the remote apply process (It allows journal entries for objects in one library to be applied to objects in another library) to support a new keyword *IGNORE. When the apply process finds these definitions it will automatically ignore any requests for objects in the defined library. NOTE:- The best solution would have been to move the developers off the production systems and develop more HA friendly approaches to making production data available, but in this case that was not an option.

We are always learning and adding new features into HA4i, many of them from customer requirements or suggestions. Being a small organization allows us to react very quickly to these requirements and provide our clients with a High Availability Solution that meets their needs. If you are looking for an Affordable High Availability or Disaster Recovery Solution or would like to ask about replacing an existing solution give us a call. We are always happy to look at your needs and see if HA4i will fit your solution requirements and budget.

Chris…

Dec 13

Linking IBMi data to a Gauge in PHP

We were thinking about how to create a new interface for one of our old utilities using PHP and decided that using a JavaScript based gauge would probably be a good start. There are plenty of free and chargeable JavaScript utilities out there that would do what we wanted, the one we settled on was a HTML5 Canvas implementation developed by Mykhailo Stadnyk. He has placed the code on the web and agreed to anyone running the code under a MIT license which basically allows you to copy and use the code where ever you want. If you would like to use the code it is available here, simply copy the javascript file into your directory structure and include.

We made a couple changes to the javascript code as it calls a remote HTTP server to pull back a font so we found a substitute font and installed it on the PC. There are a couple of examples available which can be used to demonstrate the gauge in action which again we used as the basis for our test. The only other point we should mention is that the demo does rely on HTML5 and canvas, if it is not supported in your browser the test will not work!

The method used to get the data from the IBMi is to call a Service Program through the i5_program_call available in the Aura i5_toolkit. Below is the C Program to return the information, it just calls the QWCRSSTS API and returns the Pct_Processing_Unit_Used value as can be seen in the code below.


#include <stdio.h> /* Standard I/O */
#include <stdlib.h> /* Standard library */
#include <string.h> /* String handlers */
#include <qusec.h> /* Error Code */
#include <errno.h> /* Error Num Conversion */
#include <decimal.h> /* Decimal support */
#include <qusec.h> /* Error Code */
#include <qwcrssts.h> /* System Status */
#pragma comment(copyright,"Copyright @ Shield Advanced Solutions Ltd 1998-2001")

typedef _Packed struct EC_x{
Qus_EC_t EC;
char Exception_Data[1024];
} EC_t;

int Get_Svr_Status(char *CPU_Util,char *reset) {
double Cpu_Pct;
char Reset[10] = "*NO "; /* Reset CPU % */
Qwc_SSTS0200_t Buf; /* System Status Struct */
EC_t Error_Code = {0}; /* Error Code Struct */

Error_Code.EC.Bytes_Provided = sizeof(Error_Code);
// reset the CPU Utilization?
if(*reset == 'Y')
memcpy(Reset,"*YES ",10);
QWCRSSTS(&Buf,
sizeof(Buf),
"SSTS0200",
Reset,
&Error_Code);
if(Error_Code.EC.Bytes_Available > 0) {
sprintf(CPU_Util,"99999");
return -1;
}
// Just set up the CPU Utilization here
Cpu_Pct = (double)Buf.Pct_Processing_Unit_Used / 10;
sprintf(CPU_Util,"%f",Cpu_Pct);
return 1;
}

The web page is generated using the following code.


<?php
/*
Copyright © 2010, Shield Advanced Solutions Ltd
All rights reserved.

http://www.shieldadvanced.ca/

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:

- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.

- Neither the name of the Shield Advanced Solutions, nor the names of its
contributors may be used to endorse or promote products
derived from this software without specific prior written
permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

*/
// start the session to allow session variables to be stored and addressed
session_start();
require_once("scripts/functions.php");
// load up the config data
if(!isset($_SESSION['server'])) {
load_config();
}
$conn = 0;
$_SESSION['conn_type'] = 'non_encrypted';
if(!connect($conn)) {
if(isset($_SESSION['Err_Msg'])) {
echo($_SESSION['Err_Msg']);
$_SESSION['Err_Msg'] = "";
}
echo("Failed to connect");
}
// get the information
get_cpu_util($conn);
?>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html style="width:100%;height:100%">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Gauge Test</title>
<script src="jscripts/gauge.js"></script>
<style>body{padding:0;margin:0;background:#222}</style>
</head>
<body style="width:100%;height:100%">
<canvas id="gauge"></canvas>
<div id="console"></div>
<script>
var gauge = new Gauge({
renderTo : 'gauge',
width : document.body.offsetWidth,
height : document.body.offsetHeight,
glow : true,
units : 'Cpu Utilization',
title : false,
minValue : 0,
maxValue : 110,
majorTicks : ['0','10','20','30','40','50','60','70','80','90','100','110'],
minorTicks : 2,
strokeTicks : false,
highlights : [
{ from : 0, to : 50, color : 'rgba(240, 230, 140, .25)' },
{ from : 50, to : 70, color : 'rgba(255, 215, 0, .45)' },
{ from : 70, to : 90, color : 'rgba(255, 165, 0, .65)' },
{ from : 90, to : 100, color : 'rgba(255, 0, 0, .85)' },
{ from : 100, to : 110, color : 'rgba(178, 34, 34, .99)' }
],
colors : {
plate : '#fff',
majorTicks : '#f5f5f5',
minorTicks : '#ddd',
title : '#fff',
units : '#0bb',
numbers : '#0aa',
needle : { start : 'rgba(240, 128, 128, 1)', end : 'rgba(255, 160, 122, .9)' }
}
});
gauge.onready = function() {
gauge.setValue( <?php echo($_SESSION['CPU_Util']); ?> );
};

gauge.draw();

window.onresize= function() {
gauge.updateConfig({
width : document.body.offsetWidth,
height : document.body.offsetHeight
});
};
</script>
<?php ?>
</html>

To call the Service Program we created a function ‘get_cpu_util()’ which is called every time the page is refreshed. Here is the code for the get_cpu_util function.


function get_cpu_util($conn) {
$desc = array (array ("Name" => "CPU_Util", "io" => I5_INOUT, "type" => I5_TYPE_CHAR, "length" => "5"),
array ("Name" => "Reset_Status", "io" => I5_IN, "type" => I5_TYPE_CHAR, "length" => "1") );
$prog = i5_program_prepare("PHPTSTSRV(Get_Svr_Status)", $desc,$conn);
if ($prog == FALSE) {
$errorTab = i5_error ();
echo "Program prepare failed Display_Server_Status <br>";
var_dump($errorTab);
var_dump($conn);
die ();
}
$parmOut = array("CPU_Util" => "cpu_util");
$parameter = array("CPU_UTIL" => "", "Reset_Status" => "N");
$ret = i5_program_call($prog, $parameter, $parmOut);
if (!$ret) {
throw_error("i5_program_call failed Retrieve_Status <br>");
exit();
}
// close the program call
i5_program_close($prog);
$_SESSION['CPU_Util'] = $cpu_util;

return 1;
}

The majority of the work is done in the Javascript section above with just the data extraction from the IBMi being carried out using the i5_toolkit. We have used the same connection function that we used in our other tests but instead of asking for the profile and password we added it to the config file so no sign on screen is presented before the connection is made.

The above code resulted in the following output on our systems. The actual CPU utilization differs because it is near impossible to request the page and refresh the 5250 screen at the same time. We also noticed that the returned value is always greater than the actual value shown in the WRKACTJOB screens, but the test was more about showing the gauge working using IBMi data than ensuring we saw the same data in both interfaces.

5250 Work with Active Jobs

5250 Work with Active Jobs

Gauge showing CPU Utilization %

Gauge showing CPU Utilization %

We feel this is where modernization of the IBMi and its applications should begin, making simple tools and utilities that use the IBMi data and display it in a web based interface means you can access those interfaces from many devices. We only show the output in a PC based browser but with some simple CSS and checking code you could make sure it is correctly displayed on most devices.

Happy PHP’ing.

Chris..