Semaphores to avoid critical task collision

Standard

This is a sample from my cakephp shell script that has some critical processing. It is ran every second and sometimes the script can take over a second to process. Thus, I needed a way to avoid the script from accessing the same critical business logic and manipulating data.

The semaphore works by acquiring the key with the given id: 1068. I have set the max number of semaphores to 1 for the key. All other shell scripts that are ran will autorelease when they cannot acquire the semaphore (hence another shell script is still processing).

/**
* Semaphore values
*/
private $sema_key = 1068;
private $sema_maxAcquire = 1;
private $sema_permissions = 0666;
private $sema_autoRelease = 1;

public function main() {
// Acquire our lock
   $semaphore = sem_get($this->sema_key, $this->sema_maxAcquire, $this->sema_permissions, $this->sema_autoRelease);
   if (!$semaphore) {
      throw new InternalErrorException('Error getting semaphore');
   }
   sem_acquire($semaphore);

   // DO LOGIC: CRITICAL TASK

   // Release the semaphore for other processes
   sem_release($semaphore);
}

Forking a running process and returning values

Standard

Using pcntl_fork() allows for forking of a running processing in PHP. In this example I fork my shell script and process data and return the results. This would be useful in a case where you had to make api calls to lets say 4 vendors: amazon, itunes, youtube, flickr. You would assign each child id a task to run. They each would then write there results to a shared memory location for further manipulation. I have written this test in CakePHP shell script.

class ForkShell extends AppShell {

    public function main() {

        $total = 0;
        $data = array();
        $result = array();
        $start = (float) array_sum(explode(' ', microtime()));

        echo "Parent PID: ".getmypid().PHP_EOL;

        $this->forkTest($data);

        // Manipulate the results from the fork
        foreach ($data as $key => $value) {
            $total = $total + $value['number'];
        }
        
        $result['sum_of_digits'] = $total;
        $result['data'] = $data;

        echo "Processing time: ".sprintf("%.4f", ((float) array_sum(explode(' ', microtime())) - $start))." seconds".PHP_EOL;
        print_r(Set::reverse($result));
    }

    public function forkTest(&$data = array()) {

        $pids = array();
        $parent_pid = getmypid();

        for ($i = 0; $i < 4; $i++) {
            if (getmypid() == $parent_pid) {
                $pids[] = pcntl_fork();
                echo "Forking child, now has ".count($pids)." elements".PHP_EOL;
            }
        }

        if (getmypid() == $parent_pid) {
	  		 
            // Process children results as they exit, but not before
            while (count($pids) > 0) {

                echo "Parent id: ".getmypid().PHP_EOL;

                // Wait for child to complete / maintains consistency
                $pid = pcntl_waitpid(-1, $status);
                // Open shared memory block (read only)
                $shm_id = shmop_open($pid, "a", 0, 0);
                // Read data based on chunk size to local variable
                $shm_data = unserialize(shmop_read($shm_id, 0, shmop_size($shm_id)));
                // Delete shared memory block
                shmop_delete($shm_id);
                // Close shared memory block
                shmop_close($shm_id);
                // Merge data to memory location of data
                $data = array_merge($data, $shm_data);
                /* Remove all PID entries created */
                foreach ($pids as $key => $tpid) {
                    if ($pid == $tpid) {
                        unset($pids[$key]);
                    }
                }
            }
            $pids = array();
            
        } else {

            echo "Child id: ".getmypid().PHP_EOL;

            // From here you could do API calls or other calculations and bring the results together
            $pdata = array();
            array_push($pdata, array(
                'child_id' => getmypid(),
                'number' => rand(5, 15)
            ));
            $data_str = serialize($pdata);

            // Open shared memory location
            $shm_id = shmop_open(getmypid(), "c", 0644, strlen($data_str));
            
            // attempt to write to shared memory
            if (!$shm_id) {
                exit("Couldn't create shared memory segment");
            } else if (shmop_write($shm_id, $data_str, 0) != strlen($data_str)) {
                exit("Couldn't write shared memory data");
            }

            exit(0);
        }
    }
}

I would not recommend running this script inside of CakePHP. It may cause issues with the caching.

Running shells as repeating cronjobs

Standard

It is common to use shells to handle background tasks on your server. Sometimes that task needs to be executed as quickly as possible. For example, you have a queue of tasks that need to process as soon as entered.

CakePHP can be configured on Unix servers to run a modified shell script via crontab.

Begin by modifying the defaultShell that comes with cake. I have created a new script call secondShell.

#!/bin/bash
TERM=dumb
export TERM
cmd="cake"
while [ $# -ne 0 ]; do
   if [ "$1" = "-cli" ] || [ "$1" = "-console" ]; then
      PATH=$PATH:$2
   shift
   else
      cmd="${cmd} $1"
   fi
   shift
done
COUNT=0
while [ $COUNT -le 30 ]; do
   $cmd
   let COUNT=COUNT+1
   sleep 2;
done

The modifications make this shell run ever 2 seconds for 30 occurrences (2 x 30 = 60 seconds)

To run this shell place it in the crontab by using command “crontab -e” in terminal.

*/1     *       *       *       *       /(your path)/app/Lib/Shells/secondShell YourShell -cli /usr/bin -console /path/to/your/app/Console -app /(your path)/app >/dev/null 2>&1

The */1 tells the scheduler to run the CakePHP Shell script every minute. Learn more about cron job scheduling