Aug 23

Sending emails with attachments from the IBM i

OK I have to admit I did not think of this first, I found it when I checked the latest Blog postings on iPlanet! You can find the original here. I just searched on the web to find the IBM documentation which is located here.

The reason I was really interested was due to a client issue where the iAMP server does not have any built in email function (mail()), so I was looking at how to build my own email function.

The functions I built were based on the code we produced for our HA4i product which has an inbuilt email manager for its notification process, these are written in C and use the low level socket functions to send the email directly to a SMTP server. Nothing fancy but it does work and as we are not email guru’s we thought keeping it simple was out best option. All went well until we though about adding attachments to the email, the HA4i code has no ability to add attachments because it does not need it. After a lot of reading and combing through RFC’s and Wiki pages we found the solution we needed, multipart mime was needed so we had to structure the code to allow the attachments to be correctly embedded into the email body.

After some trial and error we did get the process to work and we now have correctly formatted emails with attachments being sent from the IBM i. But we wanted to see if there are other options (we like options :-)) which is how we came across the above blog post. Running the command in a CL program etc was not what we needed, we wanted to provide a PHP version. Thankfully the i5_toolkit provides the answer, we just needed to call the command via the i5_command() function! Here is the sample code we used to test it with.

The page which is called connects to the IBM i and then uses the following to call the function

send_email_cmd($conn,"chrish@shieldadvanced.ca","This is a test message with IBM Command","/home/CHRISH/mail-1.2.0.tar");

This if the code for the function

function send_email_cmd(&$conn,$recipient,$subject,$file) {
$command = "SNDSMTPEMM RCP((" .$recipient .")) SUBJECT('" .$subject ."') NOTE('

This is the body of the email

I can enter things using HTML and format things in a most pretty way

cool') ATTACH(('" .$file ."' *OCTET *BIN)) CONTENT(*HTML)";
if(!i5_command($command,$conn)) {
echo("Failed to submit command " .$command);
}
else {
echo("Successfully sent email");
}
}

That was all there was to it! You could add a lot more code to verify the attachment types etc etc etc but our test proved the functionality does work.
Thanks to Nick for pointing out the command.

Chris…

May 16

Pagination now added to log viewer

One of the tasks we left out in the initial release of the PHP Interface of FTP Guard4i was the ability to set the page size when viewing the log entries. What we wanted to do was allow the number of log records displayed to be preset by the user, this would allow the retrieval of records to the page to be carried out a lot quicker than if all of the records were to be displayed. As part of this exercise we also decided to add a search button for data stored in certain columns of the database, this would allow you to say filter the records based on a certain object or on a certain user etc. and still provide a paged output.

The following is a sample screen where the sort parameter is the date and time column, because we provided the sort capability we do not need a search capability as well so no search box is displayed.

Paged Log View

Paged Log View

Here is a sample screen showing the sort column being the Object information and the search value was QSYS.

Paged View with Search

Paged View with Search

We are constantly looking at ways to add new features and functionality to the FTP Guard4i product, if you have any questions or would like to see a demo please let us know.

Chris…

May 06

FTP Guard4i is available for download

FTP Guard4i is now completed and available for download. We have placed the manuals online as well as the objects required to install the product. You will need to sign in as a member to download the objects and once installed you will need a key to allow the product to function. The PHP interface is available and requires the Easycom i5_toolkit functions to allow connectivity to the IBM i. We have not tested it with the Zend Free toolkit at this time and would need to make some additional changes due to the lack of support for some objects. If this is needed we can work with you to make those changes.

FTP Security is something we have been looking at for a long time, our initial requirement was highlighted because of the access to the source code for our products by the developers. We needed to give them access to the code to allow them to carry out their activities but we did not want them to be able to copy the code to other systems. The original product we created also provided an FTP Client so we could make the object transfer a lot easier than the FTP Client provided by the OS but this release only provides the security aspects required.

As part of the rewrite we have made a number of improvements in the methods we used to control the access particularly around the accept and reject IP addresses set for individual users. This allows you to set a range of IP addresses a user can connect to and from in the same manner as you can set the connection accept and reject addresses. We have also changed the logging to a Database file which allows us to add much more meaningful data about the activities carried out. While the clean up routines we have provided only allow the log to be cleared, using standard SQL against the file will provide a lot more granular entry removal.

FTP Security is an area most IBM i shops ignore because they believe the IBM i is naturally more secure than other platforms, that is not true and as we see more and more IBM i systems being linked to a wider audience we could see more intrusions being logged. FTP Guard4i also has a very comprehensive logging feature so you can now see who connects to your server and what they did while they were connected.

If you need more information about FTP Guard4i or would like to see a working demo please let us know using the demo request forms on the website.

Chris…

Apr 29

FTP Guard4i interfaces completed

We have finished the PHP interfaces for FTP Guard4i. The 5250 interfaces are going to remain pretty much the same due to the limitations set by UIM (80 columns does not fit all of the data) but we hope to eventually add some new screens once we work out what makes sense. The PHP interface uses the i5_toolkit functions to extract the data from the IBM i, this allows us to run the Apache server on a separate server which is better suited to running an Apache web server than the IBM i. We also have the same processes running under iAMP on the IBM i for testing and demonstration purposes if you wish to see a total IBM i implementation.

Here is a quick overview of the pages and the data that they show.

1. FTP Guard4i Status screen

FTP Guard4i Status

FTP Guard4i Status

The list of users who are connected to the FTP server is a new feature which is only available in the PHP interface for the initial release due to the limitations imposed by the UIM (5250) screens. We did some testing with multiple users to see exactly what users were logged in and when which provided some interesting results.
The FTP Server is the job which is listening on port 21, the SSHD Server is the job which is listening on port 22. The log writer is the job which processes all of the request events which have been created as a result of user connections, this data is stored independently so even if the log writer is not running the events will be recorded waiting for the log writer to be started. We have also listed the exit points which have been correctly registered for FTP Guard4i, if any of these exit points are inactive no FTP activity will be logged until they are reset and the FTP Server restarted.

2. FTP Guard4i Server Users

FTP Guard4i Server Users

FTP Guard4i Server Users

Access to the FTP Server can be limited in many ways, the above image shows all of the configuration aspects of the users who are allowed to access the FTP Server and what limitations if any are set for that user. You can directly control all aspects of the FTP Server activity for a particular user such as when the can connect and where from, you can determine if they can move around the library/directory structure or if they are jailed to a specific one. If a user tries to connect to a directory/library which they are not allowed they will automatically be connected to the default directory/library. The list format and Name format are set regardless of the actual FTP Server settings.

3. FTP Guard4i Client settings

FTP Guard4i Client Users

FTP Guard4i Client Users

The FTP Client which is available on the IBM i is generally open to all users, this can be a major security exposure as a user with sufficient access can link a FTP Server to the system (a PC running FileZilla Server or similar) and transfer objects off to the PC without any trace. With FTP Guard4i all FTP activity is logged and can be reviewed to see what users did when using the services. The controls provided can limit the target Server (IP Address) and what activities the user can carry out, including the directory/libraries which can be accessed.

4. FTP Guard4i Accept IP Address

FTP Guard4i Accept IP config

FTP Guard4i Accept IP list

You can set the addresses which the users can connect to the FTP Server from, this is in addition to the IP addresses which can be set in the User settings which can provide a very simple to manage access tool. The process will check for an accept address and reject address entry, if an entry matches a specific accept entry the connection will be allowed even if a reject entry matches which is less specific. The User settings are checked after the connection to verify the user can connect from the IP address after this check.

5. FTP Guard4i Reject IP List

FTP Guard4i Reject IP

FTP Guard4i Reject IP List

The above shows a single entry which states that everything is rejected which does not match an Accept entry.

6. FTP Guard4i Log

FTP Guard4i Log

FTP Guard4i Log view

The level of logging can determine what log entries are placed into the log, if it is set to log all entries you will see an entry for every request made to the server including the actual files and directories which have been involved. This can be very important for auditors who need to view all of the transactions a user carried out via the FTP Services on the IBM i.

7. FTP Guard4i Config.

FTP Guard4i config

FTP Guard4i Config

There are various control files which determine how FTP Guard4i runs, the PHP interface provides the ability to view or update those files.

As you can see FTP Guard4i is pretty much completed, all we need to do now is carry out some additional testing before we move to the release stage of the process. We will also provide a manual which will give more details on the various configuration parameters and how to manage the data which is logged.

If you are interested in FTP Guard4i and the security of the IBM i FTP Services let us know. We can provide online demos of the product and show how effective it is in locking down user FTP activities. Don’t wait until your data has been stolen, act today and give us a call.

Chris…

Apr 23

FTP Guard 4i Take 2

We had been discussing the FTP Guard 4i with a prospect and they mentioned that they would like to be able to monitor the FTP Server and SFTP Server from the FTP interface. So we have added a couple of new features to the status screen that allow the user to administer the FTP Server and the SSHD server which is used for the SFTP connections.

Here is the new status screen

New FTP Guard 4i status screen

FTP Guard 4i take 2

One of the things we did notice when we added the new features and checked they functioned was the SFTP connection takes on the QSECOFR profile in the job and drops the original user profile. We need to take a look at this to see exactly what effect this has? We don’t allow the QSECOFR profile to connect via FTP or SFTP so the security we have set for the user as far as FTP is concerned still applied.

Let us know if you are interested in this kind of solution and what if any additional features you would like to see. The Log viewer is coming along and will be the subject of our next post.

Chris…

Feb 12

Adding a Bar Graph for CPU Utilization to HA4i and DR4i

As part of the ongoing improvements to the PHP interfaces we have developed for HA4i and DR4i we decided to add a little extra to the dashboard display. The dashboard display is a quick view of the status for the replication processes, we used traffic lights as indicators to determine if the processes are running OK. Another post recently discussed using a gauge to display the overall CPU utilization for the system but we wanted to take it one step further, we wanted to be able to show just the CPU utilization for the HA4i/DR4i jobs.

We thought about how the data should be displayed and settled on a bar graph, the bars of the graph would represent the CPU utilization as a percentage of the available CPU and would be created for each job that was running in the HA4i/DR4i subsystem. This gave us a couple of challenges because we needed to determine just how many jobs should be running and then allow us to build a table which would be used to display the data. There are plenty of bar graph examples out there which show how to use CSS and HTML to display data, our only difference is that we would need to extract the data from the system and then build the bar graph based on what we were given.

The first program we needed to create was one which would retrieve the information about the jobs that are running that could be called from the Easycom program interface. We have already published a number of tests around this technology so we will just show you the code we added to allow the data to be extracted. To that end we extended the PHPTSTSRV service program with the following function.

typedef _Packed struct job_sts_info_x {
char Job_Name[10];
char Job_User[10];
char Job_Number[6];
int CPU_Util_Percent;
} job_sts_info_t;

int get_job_sts(int *num_jobs, job_sts_info_t dets[]) {
int i,rc = 0,j = 0; /* various ints */
int dta_offset = 0;
char msg_dta[255]; /* message data */
char Spc_Name[20] = "QUSLJOB QTEMP "; /* space name */
char Format_Name[8] = "JOBL0100"; /* Job Format */
char Q_Job_Name[26] = "*ALL HA4IUSER *ALL "; /* Job Name */
char Job_Type = '*'; /* Job Info type */
char *tmp;
char *List_Entry;
char *key_dta;
Qus_Generic_Header_0100_t *space; /* User Space Hdr Ptr */
Qus_JOBL0100_t *Hdr;
Qwc_JOBI1000_t JobDets;
EC_t Error_Code = {0}; /* Error Code struct */

Error_Code.EC.Bytes_Provided = _ERR_REC;
/* get usrspc pointers */
/* memcpy(Q_Job_Name,argv[1],26); */
QUSPTRUS(Spc_Name,
&space,
&Error_Code);
if(Error_Code.EC.Bytes_Available > 0) {
if(memcmp(Error_Code.EC.Exception_Id,"CPF9801",7) == 0) {
/* create the user space */
if(Crt_Usr_Spc(Spc_Name,_1MB) != 1) {
printf(" Create error %.7s\n",Error_Code.EC.Exception_Id);
exit(-1);
}
QUSPTRUS(Spc_Name,
&space,
&Error_Code);
if(Error_Code.EC.Bytes_Available > 0) {
printf("Pointer error %.7s\n",Error_Code.EC.Exception_Id);
exit(-1);
}
}
else {
printf("Some error %.7s\n",Error_Code.EC.Exception_Id);
exit(-1);
}
}
QUSLJOB(Spc_Name,
Format_Name,
Q_Job_Name,
"*ACTIVE ",
&Error_Code);
if(Error_Code.EC.Bytes_Available > 0) {
printf("QUSLJOB error %.7s\n",Error_Code.EC.Exception_Id);
exit(-1);
}
List_Entry = (char *)space;
List_Entry += space->Offset_List_Data;
*num_jobs = space->Number_List_Entries;
for(i = 0; i < space->Number_List_Entries; i++) {
Hdr = (Qus_JOBL0100_t *)List_Entry;
memcpy(dets[i].Job_Name,Hdr->Job_Name_Used,10);
memcpy(dets[i].Job_User,Hdr->User_Name_Used,10);
memcpy(dets[i].Job_Number,Hdr->Job_Number_Used,6);
QUSRJOBI(&JobDets,
sizeof(JobDets),
"JOBI1000",
"*INT ",
Hdr->Internal_Job_Id,
&Error_Code);
if(Error_Code.EC.Bytes_Available > 0) {
printf("QUSRJOBI error %.7s\n",Error_Code.EC.Exception_Id);
exit(-1);
}
dets[i].CPU_Util_Percent = JobDets.CPU_Used_Percent;
List_Entry += space->Size_Each_Entry;
}
return 1;
}

The program calls the QUSLJOB API to create a list of the jobs which are being run with a User profile of HA4IUSER (we would change the code to DR4IUSER for the DR4I product) and then use the QUSRJOBI API to get the CPU utilization for each of the jobs. We did consider using just the QUSLJOB API with keys to extract the CPU usage but the above program does everything we need just as effectively. As each job is found we are writing the relevant information to the structure which was passed in by the PHP program call.

The PHP side of things requires the i5_toolkit to call the program but you could just as easily (well maybe not as easily :-)) use the XMLSERVICE to carry out the data extraction. We first created the page which would be used to display the bar chart, this in turn calls the functions required to connect to the IBMi and build the table to display the chart. Again we are only showing the code which is additional to the code we have already provided in past examples. First this is the page which will be requested to display the chart.

<?php
/*
Copyright © 2010, Shield Advanced Solutions Ltd
All rights reserved.

http://www.shieldadvanced.ca/

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:
- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.
- Neither the name of the Shield Advanced Solutions, nor the names of its
contributors may be used to endorse or promote products
derived from this software without specific prior written
permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

*/
// start the session to allow session variables to be stored and addressed
session_start();
require_once("scripts/functions.php");
// load up the config data
if(!isset($_SESSION['server'])) {
load_config("scripts/config_1.conf");
}
$conn = 0;
$_SESSION['conn_type'] = 'non_encrypted';
if(!connect($conn)) {
if(isset($_SESSION['Err_Msg'])) {
echo($_SESSION['Err_Msg']);
$_SESSION['Err_Msg'] = "";
}
echo("Failed to connect");
}
?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Untitled Document</title>
<style>
td.value {
background-image: url(img/gl.gif);
background-repeat: repeat-x;
background-position: left top;
border-left: 1px solid #e5e5e5;
border-right: 1px solid #e5e5e5;
padding:0;
border-bottom: none;
background-color:transparent;
}

td {
padding: 4px 6px;
border-bottom:1px solid #e5e5e5;
border-left:1px solid #e5e5e5;
background-color:#fff;
}

body {
font-family: Verdana, Arial, Helvetica, sans-serif;
font-size: 80%;
}

td.value img {
vertical-align: middle;
margin: 5px 5px 5px 0;
}

th {
text-align: left;
vertical-align:top;
}

td.last {
border-bottom:1px solid #e5e5e5;
}

td.first {
border-top:1px solid #e5e5e5;
}

table {
background-image:url(img/bf.png);
background-repeat:repeat-x;
background-position:left top;
width: 33em;
}

caption {
font-size:90%;
font-style:italic;
}
</style>
</head>

<body>
<?php get_job_sts($conn,"*NET"); ?>
</body>
</html>

The above code shows the STYLE element we used to form the bar chart, normally we would encompass this within a CSS file and include that file, but in this case as it is just for demonstrating the technology we decided to leave it in the page header. the initial code of the page starts up the session, includes the functions code, loads the config data which is used to make the connection to the IBMi and then connects to the IBMi. Once that is done the function which is contained in the functions.php called get_job_sts is called. Here is the code for that function.


/*
* function to display bar chart for active jobs and CPU Usage
* @parms
* the connection to use
*/

function get_job_sts(&$conn,$systyp) {
// get the number of jobs and the data to build the bars for

$desc = array(
array("Name" => 'NumEnt', "io" => I5_INOUT, "type" => I5_TYPE_INT),
array("DSName" =>"jobdet", "count" => 30, "DSParm" => array(
array("Name" => "jobname", "io" => I5_OUT, "type" => I5_TYPE_CHAR, "length" => "10"),
array("Name" => "jobuser", "io" => I5_OUT, "type" => I5_TYPE_CHAR, "length" => "10"),
array("Name" => "jobnumber", "io" => I5_OUT, "type" => I5_TYPE_CHAR, "length" => "6"),
array("Name" => "cpu", "io" => I5_OUT, "type" => I5_TYPE_INT))));
// prepare for the program call
$prog = i5_program_prepare("PHPTSTSRV(get_job_sts)", $desc, $conn);
if ($prog == FALSE) {
$errorTab = i5_error ();
echo "Program prepare failed <br>\n";
var_dump ( $errorTab );
die ();
}
// set up the input output parameters
$parameter = array("NumEnt" => 0);
$parmOut = array("NumEnt" => "nbr", "jobdet" => "jobdets");
$ret = i5_program_call($prog, $parameter, $parmOut);
if (!$ret) {
throw_error("i5_program_call");
exit();
}
echo("<table cellspacing='0' cellpadding='0' summary='CPU Utilization for HA4i Jobs'>");
echo("<caption align=top>The current CPU Utilization for each HA4i Job on the " .$systyp ." System</caption>");
echo("<tr><th scope='col'>Job Name</th><th scope='col'>% CPU Unitlization</th></tr>");
for($i = 0; $i < $nbr; $i++) {
$cpu = $jobdets[$i]['cpu']/10;
if($i == 0) {
echo("<tr><td class='first' width='20px'>" .$jobdets[$i]['jobname'] ."</td><td class='value first'><img src='img/util.png' alt='' width='" .$cpu/2 ."%' height='16' />" .$cpu ."%</td></tr>");
}
elseif($i == ($nbr -1)) {
echo("<tr><td class='last' width='20px'>" .$jobdets[$i]['jobname'] ."</td><td class='value last'><img src='img/util.png' alt='' width='" .$cpu/2 ."%' height='16' />" .$cpu ."%</td></tr>");
}
else {
echo("<tr><td width='20px'>" .$jobdets[$i]['jobname'] ."</td><td class='value'><img src='img/util.png' alt='' width='" .$cpu/2 ."%' height='16' />" .$cpu ."%</td></tr>");
}
}
echo("</table>");
return 1;
}

The program call is prepared with a maximum of 30 job info structures, we would normally look to define this before the call and set the actual number of jobs to extract but for this instance we simply decided that 30 structures would be more than enough. After the program is called and the data returned we then build the table structure that will be used to display the data. We originally allowed the bar to take up all of the table width but after testing on our system which has uncapped CPU found that we would sometimes get over 100% CPU utilization. We still show the actual utilization but decided to halve the bar width which gave us a better display.

HA4i is running on our system in test so the CPU utilization is pretty infrequent even when we run a saturation test, but the image capture below will give you an idea of what the above code produces in our test environment.

CPU_Bar_Chart

CPU Ultization Bar Chart HA4i

Now we just need to include the relevant code into the HA4i/DR4i PHP interfaces and we will be able to provide more data via the dashboard which should help with managing the replication environment. You can see the original bar chart on which this example was produced here

Happy PHP’ing.

Chris…

Feb 07

Slow Response with i5_pconnect().


While experimenting with the latest version of our DR4i php interface we came across a slight issue with the i5_connection routines. The problem only appeared after we moved the code from our PC testing environment to the iAMP install so we thought it was simply a slow down as we moved from the PC to the IBMi, unfortunately this is only part of the problem. As soon as we found the issue we contacted Aura and asked them for support, they came back asking about how the problem was manifesting itself as they have not seen it elsewhere and were not sure what could be causing the problem.

We asked Aura about the code and what could have changed to cause the significant slow down, they said that nothing had changed and because they were not able to recreate the same issue in their network they could not understand why we were. After some further discussion and discovery they let us know that they had moved away from the gethostbyname() API to the getaddrinfo() API in preparation for IPV6 support. getaddrinfo() is the API which should be used in place of gethostbyname() API where IPV6 support is required.

We scoured the internet and found a number of entries which discussed the slowdown of lookups when getaddrinfo() was used. It was obviously a problem and we needed to understand how this was playing a part in our environment but not in Aura’s. So our first action was to write a test program which would take a host name and try to resolve that using the getaddrinfo() API. Here is the code we started off with.


#include
#include
#include
#include
#include
#include /* CEE date functions */

#ifndef NI_MAXHOST
#define NI_MAXHOST 1025
#endif

int main(int argc,char **argv) {
int error;
int junkl; /* Int holder */
double secs; /* Secs holder */
char Time_Stamp[18]; /* Time Stamp holder */
char hostname[NI_MAXHOST] = ""; /* Host name returned */
unsigned char junk2[23]; /* Junk char string */
struct addrinfo *result;
struct addrinfo *res;

CEELOCT(&junkl, &secs,junk2,NULL);
CEEDATM(&secs,"YYYYMMDDHHMISS999",Time_Stamp,NULL);
printf("Start = %s\n",Time_Stamp);
error = getaddrinfo(argv[1], NULL, NULL, &result);
/* time now */
CEELOCT(&junkl, &secs,junk2,NULL);
CEEDATM(&secs,"YYYYMMDDHHMISS999",Time_Stamp,NULL);
printf("After getaddrinfo = %s\n",Time_Stamp);
if(error != 0) {
fprintf(stderr, "error in getaddrinfo: %s\n", gai_strerror(error));
exit(EXIT_FAILURE);
}
/* loop over all returned results and do inverse lookup */
/* loop over all returned results and do inverse lookup */
for(res = result; res != NULL; res = res->ai_next) {
error = getnameinfo(res->ai_addr,
res->ai_addrlen,
hostname,
NI_MAXHOST,
NULL,
0,
0);
if(error != 0) {
fprintf(stderr, "error in getnameinfo: %s\n", gai_strerror(error));
}
if(*hostname != '\0')
printf("hostname: %s\n", hostname);
CEELOCT(&junkl, &secs,junk2,NULL);
CEEDATM(&secs,"YYYYMMDDHHMISS999",Time_Stamp,NULL);
printf("After getnameinfo = %s\n",Time_Stamp);
}
freeaddrinfo(result);
return 0;
}

When we ran this test program against our network with a simple hostname which is defined in our HOST file here is a sample of the output.

Start = 20130205095444797
After getaddrinfo = 20130205095453000
hostname: SHIELD3.SHIELD.LOCAL
After getnameinfo = 20130205095453000
Press ENTER to end terminal session.

This showed an 8 second response time for the getaddrinfo() API! Obviously this would not be acceptable as it would be used each time a connection was made. This was an issue because we do not have a DNS to resolve our local names and instead rely on the HOST table entries, our default search is set to *LOCAL so we would have expected getaddrinfo() to look up the address in the HOST table first and it would have been resolved. But due to the way the API has been coded it was always going out to the DNS server asking for an IPV6 address before looking for the IPV4 address in the HOST table.

We then looked at the documentation a lot closer and after some experimentation found that if we removed the Domain Information from the TCP/IP setup (option 12 on the CFGTCP menu) we could get the request for a server name back to immediate responses, but as soon as we added Domain information such as ‘shield3.shield.local’ the response time would instantly creep back up to over 8 seconds. Again not acceptable as the environment we needed the fix for is using NamedVirtualHosting which would always pass in a FQDN.

This is when we raised a PMR with IBM and supplied them with all of the data we had been using and asked for support. They came back with a link to a document which described the problem exactly and it was only affecting i/OS from V6R1 onwards. Because from V6R1 onwards IBM had implemented the getaddrinfo() API to do IPV6 lookups first it would always go out to the DNS for a name resolution even if an IPV4 address could be resolved from the HOST file! It would only drop back to a IPV4 lookup after the IPV6 lookup had failed!

The answer in the end was very simple, we just had to code up the AI_ADDRCONFIG flag in the getaddrinfo() request and it would only do an IPV6 lookup if more than 1 IPV6 address had been configured (::1 is not considered a configured IPV6 address). Now we see immediate responses from the API and everything works as it should even with the Domain Information configured.

If you are seeing a dramatic slowdown in your TCP/IP connection after migrating to V6R1 and you or your aplication vendor are using the getaddrinfo() API you may want to consider the above. Easycom connection routines are affected at the moment but a fix is being developed to resolve the issue.

Chris…

Dec 13

Linking IBMi data to a Gauge in PHP

We were thinking about how to create a new interface for one of our old utilities using PHP and decided that using a JavaScript based gauge would probably be a good start. There are plenty of free and chargeable JavaScript utilities out there that would do what we wanted, the one we settled on was a HTML5 Canvas implementation developed by Mykhailo Stadnyk. He has placed the code on the web and agreed to anyone running the code under a MIT license which basically allows you to copy and use the code where ever you want. If you would like to use the code it is available here, simply copy the javascript file into your directory structure and include.

We made a couple changes to the javascript code as it calls a remote HTTP server to pull back a font so we found a substitute font and installed it on the PC. There are a couple of examples available which can be used to demonstrate the gauge in action which again we used as the basis for our test. The only other point we should mention is that the demo does rely on HTML5 and canvas, if it is not supported in your browser the test will not work!

The method used to get the data from the IBMi is to call a Service Program through the i5_program_call available in the Aura i5_toolkit. Below is the C Program to return the information, it just calls the QWCRSSTS API and returns the Pct_Processing_Unit_Used value as can be seen in the code below.


#include <stdio.h> /* Standard I/O */
#include <stdlib.h> /* Standard library */
#include <string.h> /* String handlers */
#include <qusec.h> /* Error Code */
#include <errno.h> /* Error Num Conversion */
#include <decimal.h> /* Decimal support */
#include <qusec.h> /* Error Code */
#include <qwcrssts.h> /* System Status */
#pragma comment(copyright,"Copyright @ Shield Advanced Solutions Ltd 1998-2001")

typedef _Packed struct EC_x{
Qus_EC_t EC;
char Exception_Data[1024];
} EC_t;

int Get_Svr_Status(char *CPU_Util,char *reset) {
double Cpu_Pct;
char Reset[10] = "*NO "; /* Reset CPU % */
Qwc_SSTS0200_t Buf; /* System Status Struct */
EC_t Error_Code = {0}; /* Error Code Struct */

Error_Code.EC.Bytes_Provided = sizeof(Error_Code);
// reset the CPU Utilization?
if(*reset == 'Y')
memcpy(Reset,"*YES ",10);
QWCRSSTS(&Buf,
sizeof(Buf),
"SSTS0200",
Reset,
&Error_Code);
if(Error_Code.EC.Bytes_Available > 0) {
sprintf(CPU_Util,"99999");
return -1;
}
// Just set up the CPU Utilization here
Cpu_Pct = (double)Buf.Pct_Processing_Unit_Used / 10;
sprintf(CPU_Util,"%f",Cpu_Pct);
return 1;
}

The web page is generated using the following code.


<?php
/*
Copyright © 2010, Shield Advanced Solutions Ltd
All rights reserved.

http://www.shieldadvanced.ca/

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:

- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.

- Neither the name of the Shield Advanced Solutions, nor the names of its
contributors may be used to endorse or promote products
derived from this software without specific prior written
permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
POSSIBILITY OF SUCH DAMAGE.

*/
// start the session to allow session variables to be stored and addressed
session_start();
require_once("scripts/functions.php");
// load up the config data
if(!isset($_SESSION['server'])) {
load_config();
}
$conn = 0;
$_SESSION['conn_type'] = 'non_encrypted';
if(!connect($conn)) {
if(isset($_SESSION['Err_Msg'])) {
echo($_SESSION['Err_Msg']);
$_SESSION['Err_Msg'] = "";
}
echo("Failed to connect");
}
// get the information
get_cpu_util($conn);
?>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html style="width:100%;height:100%">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>Gauge Test</title>
<script src="jscripts/gauge.js"></script>
<style>body{padding:0;margin:0;background:#222}</style>
</head>
<body style="width:100%;height:100%">
<canvas id="gauge"></canvas>
<div id="console"></div>
<script>
var gauge = new Gauge({
renderTo : 'gauge',
width : document.body.offsetWidth,
height : document.body.offsetHeight,
glow : true,
units : 'Cpu Utilization',
title : false,
minValue : 0,
maxValue : 110,
majorTicks : ['0','10','20','30','40','50','60','70','80','90','100','110'],
minorTicks : 2,
strokeTicks : false,
highlights : [
{ from : 0, to : 50, color : 'rgba(240, 230, 140, .25)' },
{ from : 50, to : 70, color : 'rgba(255, 215, 0, .45)' },
{ from : 70, to : 90, color : 'rgba(255, 165, 0, .65)' },
{ from : 90, to : 100, color : 'rgba(255, 0, 0, .85)' },
{ from : 100, to : 110, color : 'rgba(178, 34, 34, .99)' }
],
colors : {
plate : '#fff',
majorTicks : '#f5f5f5',
minorTicks : '#ddd',
title : '#fff',
units : '#0bb',
numbers : '#0aa',
needle : { start : 'rgba(240, 128, 128, 1)', end : 'rgba(255, 160, 122, .9)' }
}
});
gauge.onready = function() {
gauge.setValue( <?php echo($_SESSION['CPU_Util']); ?> );
};

gauge.draw();

window.onresize= function() {
gauge.updateConfig({
width : document.body.offsetWidth,
height : document.body.offsetHeight
});
};
</script>
<?php ?>
</html>

To call the Service Program we created a function ‘get_cpu_util()’ which is called every time the page is refreshed. Here is the code for the get_cpu_util function.


function get_cpu_util($conn) {
$desc = array (array ("Name" => "CPU_Util", "io" => I5_INOUT, "type" => I5_TYPE_CHAR, "length" => "5"),
array ("Name" => "Reset_Status", "io" => I5_IN, "type" => I5_TYPE_CHAR, "length" => "1") );
$prog = i5_program_prepare("PHPTSTSRV(Get_Svr_Status)", $desc,$conn);
if ($prog == FALSE) {
$errorTab = i5_error ();
echo "Program prepare failed Display_Server_Status <br>";
var_dump($errorTab);
var_dump($conn);
die ();
}
$parmOut = array("CPU_Util" => "cpu_util");
$parameter = array("CPU_UTIL" => "", "Reset_Status" => "N");
$ret = i5_program_call($prog, $parameter, $parmOut);
if (!$ret) {
throw_error("i5_program_call failed Retrieve_Status <br>");
exit();
}
// close the program call
i5_program_close($prog);
$_SESSION['CPU_Util'] = $cpu_util;

return 1;
}

The majority of the work is done in the Javascript section above with just the data extraction from the IBMi being carried out using the i5_toolkit. We have used the same connection function that we used in our other tests but instead of asking for the profile and password we added it to the config file so no sign on screen is presented before the connection is made.

The above code resulted in the following output on our systems. The actual CPU utilization differs because it is near impossible to request the page and refresh the 5250 screen at the same time. We also noticed that the returned value is always greater than the actual value shown in the WRKACTJOB screens, but the test was more about showing the gauge working using IBMi data than ensuring we saw the same data in both interfaces.

5250 Work with Active Jobs

5250 Work with Active Jobs

Gauge showing CPU Utilization %

Gauge showing CPU Utilization %

We feel this is where modernization of the IBMi and its applications should begin, making simple tools and utilities that use the IBMi data and display it in a web based interface means you can access those interfaces from many devices. We only show the output in a PC based browser but with some simple CSS and checking code you could make sure it is correctly displayed on most devices.

Happy PHP’ing.

Chris..

Dec 07

Adding configuration Capabilities to the HA4i PHP Interfaces

We have been developing the management interface for HA4i our High Availability product in PHP for some time, but we had not got round to looking at how we could extend that interface to allow us configure the various elements of HA4i. While the existing 5250 (Green Screen) configuration panels are very effective in what they do but we wanted bring that capability into the PHP interface.

Until now, to view or update the existing configurations you needed to sign onto each system and go through each individual panel group to update the various elements that configure HA4i. We wanted to pull all of that information into a single screen where you could view the existing configuration and select a particular element to be updated.

The first panel displays the existing configuration with links (buttons) to allow you to update the particular configuration.

Here is a sample of the initial page.

New Configuration Screen

New Configuration Screen

The above screen allows each of the elements which can be updated, if a change occurs which requires the configuration to be replicated between the systems it is automatically handled in the same manner it was with the 5250 screens. We like the fact that we can now update any configuration and restart the processes all from one interface. Selecting the option to update the remote journal configuration will display a list of all available remote journals as can be seen in the sample display below.

Remote Journals available for configuration

Remote Journals available for configuration

As you can see from the display, if the journal is already configured it will be identified and a remove button provided, if it is not already configured a button allows it to be added. Because the remove button simply removes the configuration data it does not need any additional panels, but to Configure requires some of the parameters to be presented so they can be changed if required as can be seen in the sample display below.

Configure a new Remote Journal

Configure a new remote journal

Once submit is pressed on this page the remote journal is added to the configuration and the relevant objects created to allow processing by HA4i. Once this completes the list of configured remote journals is displayed again with the new journal correctly identified.

HA4i Object replication is simply a list of libraries that are to be monitored for changes, we decided to use a selection list to allow the required libraries to be configured and a list of existing configurations. You can select as many libraries as you wish in a single request which are automatically added to the configured list and all objects in the requested libraries marked for auditing so that any changes to the objects are captured and replicated. The display below shows the currently configured library plus a scrollable selection list of those libraries which are not.

Configure the libraries Object Replication

Configure the libraries for Object Replication

Output Queues have a similar interface which lists all available out queues for configuration in the same manner as the list of libraries. The following is a sample display from our test system.

Configure Outq's

Configure Outq'

The default configuration is unique to each system so there are two buttons one for each system to allow the configuration to be updated, we provided drop down options for some of the parameters to ensure the data entered is correct plus any parameters which are restricted are set to be read only. This is what our test configuration looks like.

Configure default settings

Configure default settings

That is all you need to configure HA4i, we do have a couple of filter options and IFS replication at the object level which need some attention but in the main this new interface allows you to configure, control and monitor HA4i from a single interface. The biggest gain for us was the speed at which we could implement changes using PHP and Easycom, when we developed the UIM based interfaces it would take us days just to create a single configuration interface. With Easycom and PHP we managed to build the configuration interface in just over a day.

HA4i is a premier High Availability Solution, we are constantly improving the product and interfaces making it simpler to manage and providing many new features. If you are looking at a new High Availability implementation or would like to discuss replacing an existing solution let us know, you may be surprised at just how easy and affordable our High Availability Solution can be.

Chris…

Nov 28

Capturing the return value from a function

As a C programmer I have become used to using the return values set by a function as an indicator of the success of the function to carry out its intended role, so when I created Service programs I used the same process. This raised a question about how I could capture that return value and use it within my PHP scripts as an indicator. I have read through the manuals and could not find a way to capture that value so I had resulted to creating a parameter that I would pass in and have that set on return instead of relying on the return value from the function. It seems that there is a way to capture the return value from the function, although it is not documented Easycom provides a simple way to capture the return value via a “virtual parameter”. Why is this important, well we use C a lot, so it is important that when we write a module that we can use it across multiple programs. Before we found this gem we had to write a function which we could call from our C programs and use the return value and another function with exactly the same process that used the input parameters as a way of determining the functions sucess.

Before we go whole hog into this we wanted to check to make sure that our understanding was correct, its not documented so we had to work out some of the details ourselves although Aura did give us the basic structure. The C function we will test with is a service program with a single function, it will return the index of the passed in character within an array of characters (very simple check but provided us with all we need). Here is the C program we compiled as a service Program.


#include<stdio.h>
#include<string.h>

int retvaltst(char *tstchar) {
int i = 0;
char sample[10] = "ABCDEFGHIJ";

for(i = 0; i < 10; i++) {
if(sample[i] == *tstchar)
break;
}
return ++i;
}

As you can see all it does is look at the character passed as looks for it in the sample array, if it finds the character it breaks and returns the index of the character (we increment the counter before we return it). This is the code that we used in a PHP function to test the theory, the connection and everything else is a hack of the connection routines etc we created in previous tests using the PHPTST website..


function get_ret_val(&$conn,$tstchar) {

$description = Array (array ("Name" => "tstchar", "IO" => I5_IN, "Type" => I5_TYPE_CHAR, "Length" => "1"),
array ("Name" => "retval", "IO" => I5_RETVAL | I5_OUT, "Type" => I5_TYPE_INT ));
$prog = i5_program_prepare("CHLIB/RETVALTST(retvaltst)", $description );
if ($prog == FALSE) {
$errorTab = i5_error ();
echo "Program prepare failed RETVALTST(retvaltst)";
var_dump($errorTab);
var_dump($conn);
die ();
}
$parameter = array('tstchar' => $tstchar);
$parmOut = array('retval' => 'retval');
$ret = i5_program_call($prog, $parameter,$parmOut);
if (!$ret) {
throw_error("i5_program_call failed");
exit();
}
// close the program call
i5_program_close($prog);
echo($retval);
return;
}

The function takes a character which is passed to the page as part of the call and sends it to the service program function. The set up of the program prepare shows how we set the retval parameter as a “I5_RETVAL | I5_OUT” type, Easycom knows that this is where the return value from the function is to be set. The function call only accepts a character so the i5_program_prepare() understands that the “I5_RETVAL | I5_OUT” is only used to carry the result and will not be passed into the function. We have to set the $parmOut variable to reflect the retval content as this is where it will be stored by the program. Then we call the program function and echo out the result.

Here are a couple of test results.

http://www.phptst.shield.local/retvaltst.php?tstchar=D = 4
http://www.phptst.shield.local/retvaltst.php?tstchar=F = 6
http://www.phptst.shield.local/retvaltst.php?tstchar=Z = 11 (remember we only have 10 characters so anything above 10 says we did not find it! Its case sensitive as well)

So we can now reduce the coding effort and utilize the same functions in many cases whether they are called from ILEC or PHP.. Now we need to look at the possibility of picking up other return values such as strings and structures…. There are a number of other undocumented features which we will try out in later test, particularly the ability to pass by Value or by Reference.

Happy PHP’ing.

Chris…