I already had a simple HTML page with buttons which posted login information to the login page so I could login with a single click. This page was made simple and addition websites required a new form with new hidden input fields and a button. Repeat this for each site added was becoming tedious and that is were the idea lit up.

I wanted a simple array which contains all the websites name, environment, username and password located nicely in one place. This way it’s easier to manta in the website being located in a nicely tight place. With this in mind, the following was coded.

var operators = [
    { name: "Site1", env: "Env1", username: "testuser1", password: "testpass1" },
    { name: "Site2", env: "Env2", username: "testuser2", password: "testpass2" },
    { name: "Site3", env: "Env3", username: "testuser3", password: "testpass3" },
];

With this set, I can add or remove entries at will without any real code involved.

Now it’s time for the JavaScript magic to happen. Forms all work the same, thus having a target location were to post and parameters to pass to the website. I needed a form with 2 parameters, a username and a password so it does login automatically. Yes I’m very lazy, but when managing websites all with their flavored username and password, it’s hard to remember them all. Being a developer there’s no risk in exposing passwords as the aim of this page is to be used on my computer to connect to testing servers were I’m logging in and out every minute.

jQuery came handy when building dynamic elements according to the entries in the array specified above. The code is simple, all it does is to loop through all website entries, creates a button for each and attaches a click event. The click event handles all the parameter settings, in my case, the username and password. The parameters are attached to the form dynamically and an action is set to the form according to the selected button. After all is set, the form is submitted with all parameters necessary.

$(document).ready(function() {
    for (var index in operators) {
        var operator = operators[index];
        $("<input type='button'>") // create a button
            .val(operator.env + " " + operator.name + " Site") // set button text
            .appendTo("#content") // append to main div
            .click(function() { // click event
                var operatorClicked = operators[$(":button").index(this)]; // get operator with the index of the clicked button
                $("form")
                    .append($("<input>").attr("type", "hidden").attr("name", "username").val(operatorClicked.username)) // create form username
                    .append($("<input>").attr("type", "hidden").attr("name", "password").val(operatorClicked.password)) // create form password
                    .attr("action", "http://" + operatorClicked.name.toLowerCase() + "." + operatorClicked.env.toLowerCase() + ".<<xwebsitex>>.com") // set url
                    .submit(); // submit form
            });
    }
});

The body only contained a form tag and a div were I could add the buttons. The div isn’t really necessary as i could have just added the buttons directly to the body.

This could be improved a lot by creating a more complex definition of the website object defined in the array operators. Some style should be added as well to even out all buttons on screen. For now this is what I needed and it works wonders.

If you have any suggestions on how to improve the code let me know as this is my first push at jQuery.

As discussed in part 1, using an index on the digest field brings us to a reasonable results. This post will be based around differences in speed performance between PHP vs MySQL. It is based on a lot of testing but I won’t annoy you with a lot of numbers.

First thing I noticed was, that if I increased the number of records from 500 to 1000 and even to 2000, the increase in query time was becoming drastically large. One problem was due to the index, which I’ve explained in part 1 but the difference in PHP vs MySQL was becoming evident. The table had no index for both queries which lead  me to think that MySQL version had something wrong. To cut it short, when MySQL manages to get the result spot on with the index or few records (like 10-100), the speed difference between PHP vs MySQL is negligible, but when the amount of records increase, MySQL SHA1 was really slowing down. The results I kept getting were as follows (only select queries done):

After I’ve identified that SHA1 had a problem dealing with thousands of records, I had to identify why it was slow. My first thought was that it was computing SHA1 for each record, seeing that the results above show an increase in execution time by nearly 3 times as much as PHP’s version. Considering PHP query was only doing a simple string compare for every record traversed while MySQL was doing an SHA1 function plus a string compare for each record thus slowing much more than PHP.

Deterministic or Not?

I didn’t want to accept this as SHA1 function is deterministic were if you give the same string to hash, it will always give you the same digest in return. Usually deterministic function like this, are only executed once as the result will be the same for all further checks. The results show differently and it seems it is really executing the SHA1 function for each record. The question was, how do we identify if it is really deterministic or not?

At first I thought I’d do a simple SHA1 query (ex. ‘SELECT SHA1(‘digist_string’);’) before the actual queries which would make it much more similar to the PHP version. This would be identical to PHP version but involved an extra query. A second idea had been put through one of my colleagues which was actually what I was searching for. A sub-select was the answer. In DB terms and optimizations, a sub-select is ran before and the return is used to continue the traversal bringing us to only a string comparison. So ready for another test batch with our ideal way of querying MySQL to identify if SHA1 is being used as deterministic or not was on its way.

The change in code was simple. MySQL query was changed from ”SELECT identity FROM test_table WHERE digest = sha1(?)” to “SELECT identity FROM test_table WHERE digest = (SELECT sha1(?))”. I know you may think this is babish and maybe not necessary but the results proof otherwise as the MySQL optimizer works different than our elementary reasoning. The test that follow are based on the basic table structure and a non indexed digest field. Only select queries were recorded.

As you can see from the results, MySQL is actually considering SHA1 as non-deterministic when traversing records. I’ve tried to research more about this and checked MySQL source code for this to no avail. Maybe I haven’t searched enough but till now this is my conclusion on this subject.

If you are using cryptographic functions in MySQL and the field queried is not indexed, please make sure to either change the code to a sub-query or change the structure.

Conclusion – PHP or MySQL?

Coming to the original post idea, PHP vs MySQL SHA1 functions really don’t differ in speed. My suggestion is to use what you’re mostly comfortably using and least troublesome for you. Happy programming.

Any comments or advice is appreciated.

Being the hard headed I am, I had to find out what was wrong with the tests conducted in that article. The poster kindly provided the source which was really dandy as I could execute the same tests as the poster did. After several head bangs trying to setup PHP to work flawless on my pc, I managed to analyse the tests.

At first glance, everything looked really well done and flawless. Tests were godly written and everything was up to standard. After considerable time contemplating and a beer, I came to the conclusion that the problem lies elsewhere. The tests were simple enough that it was very easy to understand and even test. One query had a fixed string and the other one had an SHA1 function call from MySQL, exactly as the test intended.

The thought that was haunting me was that even the normal PHP method, (ie. the string comparison) reading was slower than the insert. Looking at this problem alone and excluding PHP vs MySQL performance led me to the problems. Yes that’s right, there is more than one problem. It’s not how the tests are written but it’s more how databases work internally which should be taken into consideration once working with databases.

Database Search Explained

Before explaining the problem, let’s discuss a bit on how databases normally work and how they fetch data. Let’s take the tests provided by the poster and use his table as our example reference.

CREATE TABLE test_table (
    identity INTEGER AUTO_INCREMENT,
    digest VARCHAR(41),
    PRIMARY KEY (identity)
);

This is the table structure and as you can see, it is simple and perfectly normal. It has two fields, a numeric primary key and a string field of length 41 to store our hashed data.

When the database is asked to retrieve information which have ‘digest’ equals to ‘hello’, the database has to check each row if the text in ‘digest’ field matches ‘hello’. If there are 500 records, the database will check 500 records one by one and at the end returns the records which have digest = ‘hello’. As explained before, this is how normally work and not how they should work. This method of search can become very slow on large tables as checking each and every row isn’t a very intelligent way to search. Telephone directories have indices and each page is sorted by the surname to ease your searching. Imagine going through your telephone directory searching for the surname of ‘Zohan John” which should be located in the last few pages. That would take ages and this also applies for databases. This also applies for databases which make use of indices and sorting to retrieve that information quicker.

With an index on the ‘digest’ field one can locate information much faster instead of checking each field one by one. What happens when a database searches for information on an indexed field (the whole field is indexed and not the first letter or two)  is as follows. It will ask the index if there’s anything that matches the whole word ‘hello’. The index will reply with either a ‘match’ or ‘no match’. If a match is returned, the location will be identified and the database can go and retrieve the information immediately without waisting any more valuable time asking each and every record if it matches. The result is usually instant which is what raised my interest in the article.

Index Search vs Normal Search

After all the technicalities, it is time to see if what is said above is the culprit. The test is inserting 500 records into the table and querying the database 500 times one for each record which returns with a match. Without an index, the database has to go through all records and thus the ‘select’ results are slower then the ‘insert’ commands. Second beer in hand, I placed the index and the results produced were much better. The new table structure looks like this.

CREATE TABLE test_table (
    identity INTEGER AUTO_INCREMENT,
    digest VARCHAR(41),
    PRIMARY KEY (identity),
    KEY (digest)
);

This solved the first problem which was haunting me like my worst nightmare. Is ‘select’ slower than ‘insert’?

Results – Select vs Insert

My average results vary from the original article but the difference can still be identified.

This shows that the indexed search really does a great job when dealing with a lot of information. They’re a necessity for a database to work soundly. I am not implying one should go overboard with indices but one should know when and how to use them.

With this said, there’s still the performance issue between PHP vs MySQL. Could it be MySQL performs a function which basically does the same that much slower, 4 times slower? Be patient and I’ll post my finding in another post as this is already too long to read.

While I was skimming the page containing the methods and events, I got stuck on the section ‘Configuration Options’. That being said, the page looks like a JavaDoc thus placing methods, properties and the sorts on the same page and of similar format.

My first guess without reading anything I tried to set the line colors of the area chart with the following code:

var chart = new google.visualization.AreaChart(document.getElementById('chart_div'));
chart.color = [ 'black', 'green' ];

How wrong was I. To my astonishment, when I refreshed the page, the chart was gone with a JavaScript error notifying me that something wrong happened. Bite me, I went to the code again and tried a different option as the JavaScript error said that it doesn’t support that property. Fine after changing a few more options I recognized that none worked as I had thought.

Hard as my will is, I went with a different route. ‘Google supports JSON as data maybe even options?’ With that in mind, another object has been created and applied to the draw method as a parameter.

var options = {
    colors: [ 'orange', 'green' ],
    legend: 'bottom',
    title: 'Area Chart Test'
};

chart.draw(data, options);

After another refresh, the chart had re-appeared again and also significantly different. It was wider due to the legend being placed below and the colors changed as well.

With this in mind, I can say that when you work with Google tools, JSON must be a necessity to get the tools to work.

I was approached by a friend to lend a hand in his project which turns out being quite interesting. The project resulted to be simple with the end result of showing a graph of daily values.

I knew about Google visualization package which consists around a gazillion of free, very useful tools. While browsing the gallery, I encountered what i needed most: ‘Area Chart‘.

All right and on good track, I skimmed through the documentation, used the sample code and got an initial satisfactory result. So far so good. The initial problem was with how the data is inputed and from the example given:

var data = new google.visualization.DataTable();
data.addColumn('string', 'Year');
data.addColumn('number', 'Sales');
data.addColumn('number', 'Expenses');
data.addRows(4);
data.setValue(0, 0, '2004');
data.setValue(0, 1, 1000);
data.setValue(0, 2, 400);
data.setValue(1, 0, '2005');
data.setValue(1, 1, 1170);
data.setValue(1, 2, 460);
data.setValue(2, 0, '2006');
data.setValue(2, 1, 660);
data.setValue(2, 2, 1120);
data.setValue(3, 0, '2007');
data.setValue(3, 1, 1030);
data.setValue(3, 2, 540);

This really doesn’t do the job and personally it’s tedious. That fixed array value is also a pain to work with. With all the buzz of JSON these days at first glance, I managed to get the data as a simpler chunk of code.

var JSONObject = {
    cols: [
        {id: 'year', label: 'Year', type: 'string'},
        {id: 'sales', label: 'Sales', type: 'number'},
        {id: 'expenses', label: 'Expenses', type: 'number'}
    ],
    rows: [
        {c:[{v: '2004'}, {v: 1000}, {v: 400}]},
        {c:[{v: '2005'}, {v: 1170}, {v: 460}]},
        {c:[{v: '2006'}, {v: 660}, {v: 1120}]},
        {c:[{v: '2007'}, {v: 1030}, {v: 540}]}
    ]
};

var data = new google.visualization.DataTable(JSONObject, 0.6);

That is a much more like what I had in mind. With this setup, it’s much more easier to build up the data table needed to generate the chart. The object structure reference can be found here.

This concludes today’s post, but will come back and give some info on how to play with the options.

It’s 1:10 am in the morning and my eyes, body and head cannot do any more to keep me awake.

I’ll be posting some interesting coding pieces I have encountered tonight by scrambling google’s free visualisation APIs. (if I don’t forget them)

Till then, have a wonderful time.