Converting selinium code to headless tests with PhantomJS

The word test here is used ambiguously as I use selenium for more than just UI tests, it makes a great tool for browser automation which I use it for this purpose a lot. While writing your browser automation most of the time it’s easier to do it in browser mode by using the firefox or chrome driver so that you can visually inspect the HTML. Once you are done writing the code and finished testing sometimes you would prefer to switch it to headless mode so that it can be run without having a UI, at which point you are likely to tryout PhantomJS driver and your fully tested code starts throwing all sorts of errors like “Element not found” or “Stale Element Exception”. If all this errors go away if you switch back to chrome or firefox driver then the likeliest cause of your troubles is you might need to add delays in most places where you have the browser loading new data compared to the other browser drivers.
To me this seemed a bit counter intuitive at first as I though headless mode should run faster therefore requiring even less time to load UI changes, but I guess it might actually take a little longer since all the browser rendering is being done is software only. Just thought to put this out there as I have run into the issue a few times.

Automate letsencrypt certificate renewal

I recently switched from self signed certs to free SSL certs from letsencrypt and for the first time I could load my https url without getting the annoying prompt from chrome due to self signed certificates. The only problem is the certs expire pretty fast in about 90 days as of this writing, while this is nothing to complain about since the certs are free handling the renewal each time manually would be a pain and also leave me in a bind in case I forgot to do it.
I decided to automate the renewal process to save myself the hassle of having to do it manually and found two resources here and here on how to do it, I went with a combination of the two methods as my requirements were different.
I wanted the renewal to be run from a script to support email notification on success or failures which is similar to the first source and use the webroot plugin to perform renewal as it has lesser steps to perform renewal reducing any failure points during the process like the second source. The script needed to be able to run everyday and check cert expiration I didn’t want to hard code the cron job to run based on how long the certs are valid that way if letsencrypt changes the life of the certs no change is required on my side.
Let’s get started I won’t cover the install as that’s covered by letsencrypt site, I would advise you to read the different install methods and choose the one that best fits your needs.
After performing the install
Create your config file which will contain the arguments submitted to letsencrypt api I named mine “muthii.com.ini”

rsa-key-size = 4096
server = https://acme-v01.api.letsencrypt.org/directory
text = True
authenticator = webroot
agree-tos = True
renew-by-default = True
email = root@domain.com
webroot-path = /your/webserver/path

Run the command used to create/renew your certs, which creates the certs for you and shows you the path to find them.

/root/.local/share/letsencrypt/bin/letsencrypt -c /path/muthii.com.ini -d muthii.com -d www.muthii.com auth

Only run the above command if you haven’t created your certs or are ready to renew your current certs, otherwise just grab the script file and add it to your cron. Make sure to change the emails and file paths based on your setup. I have commented out the echo statements and only enable then for testing

For someone doing this for the first time locate your ssl.conf file used by your server and set the paths to the new certs

SSLCertificateFile /etc/letsencrypt/live/domain.com/cert.pem
SSLCertificateKeyFile /etc/letsencrypt/live/domain.com/privkey.pem
SSLCertificateChainFile /etc/letsencrypt/live/domain.com/fullchain.pem

Once you are done setting up head over to SSLLabs and test your certificate is recognized as expected, then setup a cron job to run the script daily .

0 2 *  * * sh /path/SSLRenew.sh

CIFS VFS: cifs_mount failed w/return code = -2

I hit this error while adding a samba mount to my fstab, but mounting the same end point would work when executed from command line. For my scenario it turns out that it might be an issue with cifs-utils or kernel if your mount point is under more than one sub-directory. My solution was to go with option 3 and have my target as a share

"//host.IPAddress/share/subdir/subdir/target" - This failed with error "CIFS VFS: cifs_mount failed w/return code = -2"
"//host.IPAddress/share/target" - This worked
"//host.IPAddress/target" - This worked

Owncloud Error: Console has to be executed with the same user as the web server is operated

After a recent Owncloud 8.o.x update I started getting this error being logged whenever the owncloud cron job run. To resolve the issue I had to change the cron job to be run as the user apache.

su -s /bin/sh apache -c  "php -f /path/to/owncloud/cron.php"

The webserver on CentOS is run under the user apache, on other linux flavous it’s www-data to find out what it is on your system just check the error being logged it will log the user running the webserver.

Console has to be executed with the same user as the web server is operated
Current user: someuser
Web server user: apache <- This is the user you want.
Unexpected error!

FreeNX – 1004 Error: NX Agent exited with exit status 1

This error is very generic and while googling I found different issues can cause it. I was able to resolve it this particular instance by creating the folders

/tmp/.X11-unix - as root
/tmp/.ICE-unix - as user logging in

and file

/tmp/.X0-lock - as root

Which had been deleted while manually cleaning up a previous session. The statement below was also logged when this error occured but it to appears to be a generic error logged for different cases whenever a session fails.

596 Session startup failed

KVM usage tips

I have been playing around with KVM lately, just to see how it stacks up against VirtualBox and am loving it. I have found the following commands interesting while working with VMs.

List all VMs

virsh list --all

Start stop a VM

virsh start/stop vmName

Clone VM

virt-clone --original vmName --name newVmName --file /some/path/newVmName.qcow2

Clone VM and assign Random Mac address to new image

virt-clone --original vmName --name newVmName --file /some/path/newVmName.qcow2 --mac=RANDOM

Copy over the xml for the new clone which you can use to define your clone if moving the image to another host

cp /etc/libvirt/qemu/newVmName.xml /some/path/newVmName.xml

Shrink/Sparcify VM image

virt-sparsify --format qcow2 --compress vmName.bkp.qcow2 vmName.qcow2 --tmp /some/path

This command assumes you renamed the original VM image file to “vmName.bkp.qcow2” incase the new image has problems you can revert to original image. You can skip the –tmp argument if you have more than double the size of the VM image available in your /tmp directory. You will need to make sure you have package “libguestfs-tools” installed which supplies the sparsify command.

Delete VM

virsh undefine vmName

This removes the XMl configuration for qemu you might have to delete the VM image yourself.

List storage pools

virsh pool-list

Refresh pool data

virsh pool-refresh poolName

You will need to run this command if you delete a VM and you need to reuse the VM name you had used, so make sure you refresh the pool where your deleted image was located.

Move VM to another system Offline
Step 1 from original host shutdown your VM, copy xml and image to backup location.

virsh dumpxml vmName > /bkpLocation/vmName.xml
cp /locationofVMs/vmname.qcow2 /bkpLocation/vmName.qcow2

Bye default if you didn’t change image storage location it will be “/var/lib/libvirt/images”. I prefer to store them in a separate partition from the OS.

Step 2 on new host get VM xml and image and enjoy.

virsh define /bkpLocation/vmName.xml
Domain vmName defined from /bkpLocation/vmName.xml
virsh start vmName

This all works if you are moving from host1 and host2 with similar cpus, for my purpose host2 had a different cpu type so I had to first create a new vm in host2 and compare the xml with my backed up copy then changed a few lines specific to CPU on host2.

Disable sleep keyboard button Linux

Recently I needed to disable the sleep button on my Fedora desktop. At first I was going to use

xmodmap

but the main drawback with this method is that changes made are cleared on reboot so I would need to create a script to be run at boot to maintain my settings.
A little more googling and I found a better way of performing the change using

pklocalauthority

. By creating a file “/etc/polkit-1/localauthority/50-local.d/somefile.pkla” having extension .pkla with the following entries:

[Disable suspend]
Identity=unix-user:*
Action=org.freedesktop.upower.suspend
ResultAny=no
ResultActive=no
ResultInctive=no

[Disable hibernate]
Identity=unix-user:*
Action=org.freedesktop.upower.hibernate
ResultAny=no
ResultActive=no
ResultInctive=no

This has been tested successfully on Fedora 20, my centos box doesn’t need this as sleep is blocked by default.

FYI: Some xmodmap commands i found useful when researching the issue.
To list all keycode mappings

 xmodmap -pke

To disable keyboard mapping for sleep”X86Sleep” button on my system which is mapped to “keycode 150”.

xmodmap -e 'keycode  150 = NoSymbol'

Update:
After second reboot KDE ignored the localpolicy file and I had to reconfigure the sleep button within KDE. I can’t explain why it worked after first reboot, but for now am just going with redefining the button in KDE.

Eclipse crashes after update to CentOS 6.6

I updated my centOS to the lates update 6.6 and eclipse started crashing every time I tried to use the editor. At first I thought it was a SELINUX issue but this was happening even after turning off SELINUX. Turns out it’s a bug with kepler and how it checks for installed Cairo version. Launching eclipse from console will give you the error below.

eclipse: cairo-misc.c:380: _cairo_operator_bounded_by_source: Assertion `NOT_REACHED’ failed.

This issue is fixed in Luna if you want to upgrade or you can disable Cairo in eclipse.ini with the command below to keep using your current eclipse version.

-Dorg.eclipse.swt.internal.gtk.cairoGraphics=false

Thanks to @Riley for the workaround.

Love OOP Python

Python makes it easy to automate manual tasks. Over time I have accumulated some scripts each for a different task but not sharing any code between them but having some similar logic, basically a few of them were having duplicate logic. So this week as I was adding another script to download and add the latest plex media server to my local repo, I came over an article about OOP python on Stackoverflow.
After creating a class for all the common logic in my scripts I was able to reduce the size of most of them by over half, I make my living writing OOP code for java but OOP for python was new to me. I wish there was OOP for bash 🙂 I have a bunch of bash scripts that need culling.

Cloud heaven = Plex + Android + CloudSync + Linux server + Own Cloud

I recently started hosting my own cloud service and it has been an awesome experience. After having my smart phone for sometime i started running low on storage, while I had backed up some pictures to a local share this is not really convenient. We take pictures on our phones so that we can have them close by so having to be at home to view my photos and videos was not very interesting. With a few apps and a Linux server I was able to put together a cloud service for the whole family that we are all loving very much.

The setup
Own Cloud installed on the Linux server. I tried a couple own cloud plugins for multimedia and none worked out for me. Using own cloud’s android sync app I ran into issues as it would lock up my phone when syncing. To fix this two issues I decided to have Plex handle the multimedia stuff and CloudSync handle the syncing process. Since I already had Plex serving up my video content adding photos was a breeze, plex also displays all your photos as thumbnails on your phone which you can click to view while the own cloud app displays a file name list which you have to click on a file to view it. I installed cloudSync on all the phones that needed backing up, this is like the swiss knife of sync apps it supports several cloud services plus regular services such as sftp,ftp and webdav just to name a few. You can back up your data to different locations by setting up filters for each data type which is awesome as I don’t want pictures and videos in one folder.
With this setup am able to have pictures and videos synced instantly and rsynced between different storage clients for redundancy. Within minutes of any of any of the phones taking a picture or a video any of the other phones can view that content so no need to send each other whatever you just captured.

While regular cloud providers offer you a few gigs for free and you have to pay once you get over 20gigs by hosting your own cloud you have terabytes of data readily accessible under your own control. You can use this cloud providers for just backing up your sensitive documents that you need an offsite backup in case of fire etc but remember to encrypt it first.

So now with everything backed up I have cleared my phones storage barely using a quarter of it and I still have access to all the pictures and clips that have been taken by any phone in the house for the last few years.