Nov 30

Time taken for Password encryption

One of the recent posts showed how to connect to a remote IBM i system and store the password in a secure method in the session variables. As part of another development project we decided to add the process to the connection code to see what effect it would have. The time taken to make the connection and display the data seemed to take a much longer time. Not sure if the time was due to the data being collected or due to the connection process we decided to write a test page to see the actual time it took. The results were quite shocking, in effect the encryption process increased the connection time by a factor of 3. This meant every time we loaded a page of data it was 3 times slower than if we just did a normal unencrypted connection.

The connection is the same for every request so the additional time can only be associated with the time it took the system to encrypt and decrypt the password. The encryption is only done once on login but decryption is done every time a new set of data was requested. To prove the point we created a sample page that would show a login screen if no connection had been made and show the connection time if the connection was made. its a very simple page and the functions that back it up are just as simple. The encryption process is the same as published in the post mentioned above so it you want to see those functions go to the post.

Here is the page we used for the initial sign on.

<?php 
/*
Copyright © 2010, Shield Advanced Solutions Ltd
All rights reserved.

http://www.shieldadvanced.ca/

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:

- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.

- Neither the name of the Shield Advanced Solutions, nor the names of its
contributors may be used to endorse or promote products
derived from this software without specific prior written 
permission. 

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN 
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 
POSSIBILITY OF SUCH DAMAGE.

*/
// start the session to allow  session variables to be stored and addressed
session_start();
require_once("scripts/functions.php");
if(!isset($_SESSION['server'])) {
	load_config();
	}	
?>

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Test PHP</title>
<!-- add the main CSS -->
<link rel="stylesheet" href="css/tst.css" type="text/css">
</head>

<body>
<?php
// if valid user is set connect using Private connection
if(isset($_SESSION['valid_usr'])) {
	$start_time = microtime();
	$conn = 0;
	if(!connect($conn)) {
		if(isset($_SESSION['Err_Msg'])) { 
			echo($_SESSION['Err_Msg']); $_SESSION['Err_Msg'] = "";
			}
		} 
	else {
		echo("Connected to " .$_SESSION['server'] ."using " .$_SESSION['conn_type']); 
		$end_time = microtime();
		$wire_time= control_microtime_used($start_time,$end_time)*1000000;
		printf("<br>total=%1.2f sec ",round($wire_time/1000000,2));
		echo("<br><a href='scripts/logout.php'>logout</a>");
		}
	}
else { ?>
        <form name=login method="post" action="scripts/login.php">
            <table width="20%" align="center" border="1" cellpadding="1">
                <tr><td><label>User ID :</label></td><td><input type="text" name="usr" /></td></tr>
                <tr><td>Password:</td><td><input type="password" name="pwd" /></td></tr>
                <tr><td colspan="2"><?php if(isset($_SESSION['Err_Msg'])) { echo($_SESSION['Err_Msg']); $_SESSION['Err_Msg'] = ''; } ?></td></tr>
                <tr><td><label>Connection type</label></td><td><select id="conn_type" name="conn_type"><option value="encrypt">Encrypted</option><option value="decrypt">Decrypted</option></select></td></tr>
                <tr><td colspan="2" align=center><input type="submit" value="Log in" /></td></tr><?php
                if(isset($_SESSION['Pwd_Err'])) {
                    if($_SESSION['Pwd_Err'] == 1) { ?>
                        <tr><td colspan="2" align=center>Sorry the credentials were rejected by the <?php echo($_SESSION['sys']); ?> System</td></tr><?php
                        $_SESSION['Pwd_Err'] = 0;
                        } 
                    } ?>
            </table>
        </form><?php
	} ?>
</body>
</html>

If the user information is correct and the connection is made the sign on screen is no longer displayed and the connection information is displayed.

This is the login script.

<?php 
/*
Copyright © 2010, Shield Advanced Solutions Ltd
All rights reserved.

http://www.shieldadvanced.ca/

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions
are met:

- Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.

- Neither the name of the Shield Advanced Solutions, nor the names of its
contributors may be used to endorse or promote products
derived from this software without specific prior written 
permission. 

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS
FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 
COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES INCLUDING,
BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; 
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 
LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN 
ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE 
POSSIBILITY OF SUCH DAMAGE.

*/

// allow session variables
session_start();
$conn = 0;
require_once"functions.php";
// make sure they are not trying to sign on with a system profile
$prf = strtoupper($_POST['usr']);
if(($prf == "QDBSHR") ||
	($prf == "QDOC") ||
	($prf == "QLPAUTO") ||
	($prf == "QLPINSTALL") ||
	($prf == "QRJE") ||
	($prf == "QSECOFR") ||
	($prf == "QSPL") ||
	($prf == "QDFTOWN") ||
	($prf == "QTSTRQS") ||
	($prf == "QSYS")) {
	$_SESSION['Err_Msg'] = "Cannot use user profile " .$_POST['usr'] ." for connection";
	header('Location: /index.php');
	exit(0);
	}
$_SESSION['usr'] = $_POST['usr'];
if($_POST['conn_type'] == 'encrypt') 
	e_pwd($_POST['pwd']);
else
   $_SESSION['pwd'] = $_POST['pwd'];	
$_SESSION['conn_type']  = $_POST['conn_type'];
// if failed to connect set the $_SESSION variables to empty
if(connect($conn) == -1) {
	$_SESSION['Pwd_Err'] = 1;
	$_SESSION['usr'] = "";
	$_SESSION['pwd'] = "";
	$_SESSION['valid_usr'] = NULL;
	header('Location: /index.php');	
	exit(0);
	}
// connect set the required session variables so just set the valid user variable	
$_SESSION['valid_usr'] = $_POST['usr'];	
header('Location: /index.php');	
exit(0);
?>

As you can see it is pretty simple in that it just determines the selected connection type and then calls the connect, if it succeeds it sets the relevant session variables.
For the connect function we have the following code

function connect(&$conn) {
// reset the the ErrMsg variable
unset($_SESSION['ErrMsg']);
// connect to the i5
$conId = 0;
if (isset($_SESSION['ConnectionID'])) {
		$conId = $_SESSION['ConnectionID'];
		}
$server = $_SESSION['server'];	
$addlibl = array($_SESSION['install_lib']);
// options array for the private connection
$options = array(
      			I5_OPTIONS_PRIVATE_CONNECTION => $conId,
      			I5_OPTIONS_IDLE_TIMEOUT => $_SESSION['timeout'],
      			I5_OPTIONS_JOBNAME => 'PHPTSTSVR');
// connect to the system  
if($_SESSION['conn_type'] == 'encrypt')     			
	$conn = i5_pconnect($server,$_SESSION['usr'],d_pwd($_SESSION['pwd']),$options);
else
	$conn = i5_pconnect($server,$_SESSION['usr'],$_SESSION['pwd'],$options);
// if connect failed
if(is_bool($conn) && $conn == FALSE) {
	$errorTab = i5_error();
	if ($errorTab['cat'] == 9 && $errorTab['num'] == 285){
		$_SESSION['ConnectionID'] = 0;
  		$_SESSION['Err_Msg'] = "Failed to connect";
  		return -1;
 		} 
 	else {
 		//set the error message
 		$_SESSION['Err_Msg'] = "Connection Failed " .i5_errormsg();	
		// send back to the sign on screen 
		$_SESSION['ConnectionID'] = 0;
 		return -1;
 		}	
	}
return 1;
}

The only other function we use are the microtime() function and a function which calculates the time taken between the start and end time.
Here is the page information which was collected for a unencrypted connection.

Connected to shield3using decrypt
total=0.13 sec

and here are the results when we connect using encryption for the password.

Connected to shield3using encrypt
total=0.38 sec

As you can see it is nearly 3 times as long to make the connection when encrypting the password. So our challenge now is to see if we can find a better or should I say faster encryption method to use. If the connection is not important because the data is refreshed too often it should not be an issue especially when you consider just how much better the security is for the stored passwords, but you will have to weight up that with the user experience for slow connections.

If you can see anything we did wrong let us know and we will happily re-code and try it again. Sorry about the code layout but it is pretty hard to show correctly formatted code on the blog setup we have.

Chris…

Nov 30

New PHP interface for JobQGenie underway

After a slow start we have now started to create a PHP based interface for the JobQGenie product. The problem we were struggling with was how to best display the list of jobs that JobQGenie has captured information for. In some customer environments this could be millions of jobs so the data had to be paged in and out efficiently. We also wanted to be able to move from one page to the other using links which we would store at the bottom of the list.

The initial test showed promise and we could effectively display the list of jobs and page through them without any issues. But we then had to add filtering, obviously this would require a number of filters such as job state or which job queue they ran on or who ran them, this is where the fun started. Our initial SQL code would fetch all of the records and display them as we needed, the order of the jobs listed was controlled by a time stamp, so a sort using another value would effect how the next set of entries would be retrieved. This is because the key on the file is by a time stamp of when the job was loaded, if I sorted the list by lets say the user name time stamps would be moved to further down in the list as the user name was the list content. This meant I could not retrieve the time stamp and list from this time stamp onwards before sorting by user name, the sorted list cause a number of entries to be missed.

I had seen some examples in the IBM manuals using the WITH clause, this would create a pre-selected list which I could order in the manner I needed and then select from that list using the time stamp as the key. This is because the time stamp would still be in the right place in the pre-selected list. Unfortunately this did not work, for some reason the resulting list was missing a number of entries. We are not sure why (probably a coding issue) but it seemed we had hit a brick wall. I decided to post a question on the forums to see if anyone had any better ideas. Fortunately we got an answer from one of the forum members, he suggested we use the RANK() function to help us create the list in the correct format, this required a unique key to base the request on which was pretty easy to arrange. So after some trial and error we finally found a SQL string which met all of our needs and provides us with a complete and sorted list and allows us to successfully page through.

Here is the actual code which generated this list. The BETWEEN numbers and the ORDER BY values are computed by the program based on the selection made by the user, this is only the initial request and subsequent requests would change the BETWEEN numbers. As far a speed is concerned we saw no real degradation to our initial method so we hope its pretty efficient.


WITH SubQ as (
SELECT
JOBNAME,
USRNAME,
JOBID,
JOBQ,
JOBQLIB,
unstamp(ENTERTS) as TS,
unstamp(STARTTS) as STR,
unstamp(ENDEDTS) asEND,
JOBTYPE,
SUBTYPE,
ENDCDE,
PRCUSED,
JOBSTATE,
RANK() OVER(ORDER BY JOBID) as RN FROM JG_SHIELD2/JQ0 where
JOBSTATE = '2' ORDER BY JOBID )
SELECT * FROM SubQ WHERE RN BETWEEN 1 AND 20

Here is a sample of the output, you will notice we have a drop down box for one filter which allows us to filter by job state, the other filter is a Javascript function attached to the cell on the header row. Eventually we will add arrows or some other image to show the column selected for the sort but for now a simple text label will do.

PHP_JobQGenie-job-list

PHP_JobQGenie-job-list

Once you have built the list you can also display the detail for the job as shown below.

PHP_JobQGenie-job-detail

PHP_JobQGenie-job-detail

This is still early days as we have to build new functions to allow searches against the job list and allow jobs to be re-submitted from the browser list but we have what we felt was the hardest part completed and Bryan who gave us the pointers certainly helped move that forward a lot faster than we would have by reading the manuals.

Now we can concentrate on getting the initial interface designed and coded before we look at the search requirements and job re-submission.

If you are interested in JobQGenie take a look at the products page, its gaining a lot of popularity as people start to understand just how exposed there High Availability environments are without this kind of solution.

Chris…

Nov 11

New tool to delete logical files based over a physical file before deleting the physical file


One of the problems I see a lot is where a physical file has a number of logical files built over it and you try to delete it only to find out you have to delete all of the logical files first. The same goes for members so we decided to create a tool that will look for the logical file members linked over the physical file member and delete them before deleting the physical file member. If you enter *ALL for the members it will remove all of the members plus the file object, if you pass in a specific member it will only remove the logical file members and that specific member in the physical file. We found we could have a physical file with no members that had logical files built over it! The API we use to extract the member list threw a fit (it found the logical files but said in the header it did not?? that’s one for IBM when we get the time).

The tool is available for download from the Blog downloads.

Chris..

Nov 07

DR4i Version 7.1 available

A new version of DR4i has been released, it brings the improvements developed for the HA4i process into the DR4i product plus adds the support for data area creation V6R1 of the OS provides. This version of DR4i now requires V6R1 as a minimum, the previous version of DR4i will still remain for those few customers who are using it on V5R4 of the OS.

Chris…