Today, I thought we’d do a followup post to this one where we initially started looking at shell scripts. Exports are by default, single-threaded, i.e., you get 1 file exported out and when it hits the maximum (2 GB in most cases), a separate file is spawned. But what if we wanted to do multi-threaded exports? MaxL allows you to do this and it reduces the time needed to export out large databases. To do this with our shell script structure, let’s make a simple update to our definition file, by adding one more column at the end.
The third column, is intended to act as a thread count. In other words, the “CUBE” database should export to 1 file, while “CUBE2” should export 2 files. Now I’ve also made a copy of the original MaxL file from earlier.
The second one is identical except that it exports 2 threads:
|
1 |
export database $2.$3 Level0 data to data_file "$5//$2_$3_1.txt","$5//$2_$3_2.txt"; |
We also need to change up the “while” loop in our shell script.
|
1 2 3 |
while IFS=',' read -r APP DB <strong>THREADS</strong>; do sh $MAXLSH/startMaxl.sh -D $MAXLDIR/Nightly_Export_<strong>$THREADS.maxls</strong> $KEY $ESS_SERVER $APP $DB $LOGDIR/Essbase_Daily $HOMEDIR/Export/ESSBASE_LEV0_DATA/${DATESTAMP}_Export 2>&1 >> $ESSBASE_DAILY_LOG done < $DEFDIR/ESSBASE_BACKUP_2.txt |
We’ve added a “THREADS” parameter which tells the script which MaxL file to call. Now if we run the script again, we get our desired output.
The script in it’s entirety looks like this:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
#!/bin/sh ## Set variables HOMEDIR=/App/Admin DEFDIR=$HOMEDIR/Definitions LOGDIR=$HOMEDIR/Logs ERRORDIR=$HOMEDIR/Errors MAXLDIR=$HOMEDIR/MAXL DATESTAMP=`date +"%Y_%m_%d"` ESSBASE_DAILY_LOG=$LOGDIR/${DATESTAMP}_Essbase_Daily.log APPDIR=/App/Oracle/Middleware/user_projects/epmsystem1/EssbaseServer/essbaseserver1/app MAXLSH=/App/Oracle/Middleware/user_projects/epmsystem1/EssbaseServer/essbaseserver1/bin KEY=213367771,507947599 ESS_SERVER=UNLK.CUBE.COM ## Create directories echo "*******************************************************" >> $ESSBASE_DAILY_LOG echo Essbase Backup Started on `date` >> $ESSBASE_DAILY_LOG echo "*******************************************************" >> $ESSBASE_DAILY_LOG echo -e '\n' >> $ESSBASE_DAILY_LOG mkdir -pv $HOMEDIR/Export/ESSBASE_LEV0_DATA/${DATESTAMP}_Export >> $ESSBASE_DAILY_LOG echo -e '\n' >> $ESSBASE_DAILY_LOG echo "Export process begins." >> $ESSBASE_DAILY_LOG echo -e '\n' >> $ESSBASE_DAILY_LOG ## Loop through and create exports while IFS=',' read -r APP DB THREADS; do sh $MAXLSH/startMaxl.sh -D $MAXLDIR/Nightly_Export_$THREADS.maxls $KEY $ESS_SERVER $APP $DB $LOGDIR/Essbase_Daily $HOMEDIR/Export/ESSBASE_LEV0_DATA/${DATESTAMP}_Export 2>&1 >> $ESSBASE_DAILY_LOG done < $DEFDIR/ESSBASE_BACKUP_2.txt echo -e '\n' >> $ESSBASE_DAILY_LOG echo "Export process ends." >> $ESSBASE_DAILY_LOG echo -e '\n' >> $ESSBASE_DAILY_LOG echo "*******************************************************" >> $ESSBASE_DAILY_LOG echo Essbase Backup Ended `date` >> $ESSBASE_DAILY_LOG echo "*******************************************************" >> $ESSBASE_DAILY_LOG echo -e '\n' >> $ESSBASE_DAILY_LOG exit |
And there you have it. A simple example of how to use shell effectively to keep our scripts lean. As you can imagine, this type of approach can be helpful in a number of scenarios. More parameters could be added to the definition file to control other aspects of your maintenance process.
