Python, PHP and a Pet Project

php-code-1242330Over the past month or so I’ve been working on a personal pet project (on my own time) to parse Cisco IOS configurations. My project goal is to provide output based on a Cisco IOS configuration to help increase my own efficiency for some troubleshooting and configuration tasks at work and also possibly help others with less experience more easily sift through some of the complexities of the configuration and understand the relationships between the various elements.

In addition to the project goal stated above, I Initially was also using this project to learn Python. Once I got the basics of Python down (I really like the language) I made some solid progress, largely thanks to the outstanding ciscoconfparse Python package. However, I eventually wanted to make this project available online via this site so others could possibly benefit from the work but I couldn’t get the ciscoconfparse package to integrate with my hosting provider system even after wasting more time than I’m willing to admit or should have trying (it works fine in my development environment, the problem has something to do with Python, security and permissions on a shared host). So I’m keeping Python and the ciscoconfparse package in my technical tool belt, but I decided to switch to PHP for this project.

Switching to PHP for this project means I don’t have a nice pre-built package to provide the base framework to parse Cisco configurations and so I’ll have to build that myself. Frankly I feel my PHP skills have become stale and so this presents a good “write erase” opportunity to refresh those and learn about and use object oriented programming.

To get up-to-speed I’ve been reading through “PHP in a Nutshell” along with some other books via my Safari Books Online account (a highly recommended resource), experimenting along the way. As far as environment goes I’m using a CentOS 6.6 virtual machine with a standard LAMP base installation as my development server and a laptop running Windows 8.1 for writing code. NuSphere PhpED had everything I was looking for and more in a PHP IDE (Integrated Development Environment) and has proven to be very helpful in my development efforts.

I’ve made some headway and at a point where I want to be able to maintain revision control. Until today I had been manually taking periodic snapshots of the project files so I could recover if I broke something that had previously been working. This was quite cumbersome so I did a little research on revision control systems and settled on Mercurial since it seemed simple and appropriate for my intents and purposes. The installation was ridiculously easy on my CentOS server.

# yum install mercurial

Once Mercurial was installed I created a basic .hgrc file in my home directory on my development VM.

[ui]
username = Walter Streeter <MY@EMAIL.ADDRESS>

After that the “Lone developer with nonlinear history” section of the Mercurial guide provided instruction for usage. The basic process I used is listed below including comments.

$ cd /path/to/projects/ # cd to where I keep my project hierarchy
$ hg init projectname # in my case projectname already existed with files
$ cd projectname
$ (add files) # in my case projectname already existed with files
$ hg add # told Mercurial to track all files
$ hg commit -m "Initial code base" # committed existing files
$ (made some changes) # made changes in a code file in PhpED and uploaded
$ hg diff # noted changes
$ hg commit -m "" # saved changes
$ hg log # noted history of changes
$ hg update 0 # reverted back to original code base
$ cat changedfile # confirmed file reverted
$ hg cp # note that hg should be used to copy files or folders
$ hg mv # note that hg should be used to move files or folders

That’s it for this post. I thought some others might benefit from what I’ve found and experienced so thought I’d share it. Now back to my personal pet programming project for some weekend fun doing “nerd stuff” as my wife would say.

Goodbye IPPlan, Hello phpIPAM!

After many years of working together I’ve had to say goodbye and retire IPPlan. The open-source IP address management (IPAM) software has served me, my employers and many others well for many years. However, it hasn’t been updated in years and has really started to show its age both in appearance and functionality. After searching for and trying other open-source options I found a younger, fresher alternative that I really like. While there are a number of options available, I chose and have migrated to phpIPAM (http://phpipam.net/) because it boasts the many features noted below including IPv4 and IPv6 support, flexible organization and easy AD (Active Directory) integration.

  • Section / Subnet separation
  • Subnet nesting
  • IPv4/IPv6 support
  • Subnet ICMP/telnet scanning and automatic status updates
  • Displays free range and number of clients
  • Subnet statistics
  • User management
  • AD/LDAP/OpenLDAP authentication support
  • E-Mail notification with IP details
  • Import IP addresses from XLS / CSV file
  • Export IP database to XLS file
  • IPv4/IPv6 calculator
  • Search IP database
  • IP request module

Below is the process I had to go through to get it installed and functional using a CentOS 7 operating system. There were some issues encountered along the way that took some troubleshooting to resolve which is why I thought I’d share my (edited) installation notes. Hopefully this post provides encouragement install and try out phpIPAM and provides some help with that installation.

CentOS 7 phpIPAM Installation Process

    • Created a new VM with 80 GB of storage.
    • Installed CentOS 7, sorry, I can’t remember what options I used, probably web server related though
    • Connected to VM via SSH, sudo as root.
    • Started httpd.
systemctl start httpd
    • Set httpd to start at boot.
systemctl enable httpd
    • Configured firewall to allow http/https.
# firewall-cmd --zone=public --add-port=80/tcp --permanent
success
# firewall-cmd --zone=public --add-port=443/tcp --permanent
success
# firewall-cmd --reload
success

See: http://linuxconfig.org/how-to-open-http-port-80-on-redhat-7-linux-using-firewall-cmd

    • Installed VMware tools.
mount /dev/cdrom /mnt
cp /mnt/VMwareTools-9.4.6-1752774.tar.gz .
tar -xvzf VMwareTools-9.4.6-1752774.tar.gz
    • Installed MariaDB (mysql) server.
yum install mariadb-server
    • Started MariaDB and set to start at boot.
systemctl start mariadb.service
systemctl enable mariadb.service
ln -s '/usr/lib/systemd/system/mariadb.service' '/etc/systemd/system/multi-user.target.wants/mariadb.service'
    • Installed some additional php modules.
yum install php-mysql php-xml php-xmlrpc php-soap php-gd php-ldap php-mbstring php-odbc php-pear
    • Restarted httpd.
systemctl restart httpd
    • Checked versions on all installed.
# php -v
PHP 5.4.16 (cli) (built: Oct 31 2014 12:59:36)
Copyright (c) 1997-2013 The PHP Group
Zend Engine v2.4.0, Copyright (c) 1998-2013 Zend Technologies
# httpd -v
Server version: Apache/2.4.6 (CentOS)
Server built: Mar 12 2015 15:07:19
# mysql -V
mysql Ver 15.1 Distrib 5.5.41-MariaDB, for Linux (x86_64) using readline 5.1
    • Created a phpinfo script, tested http connection to the server, reviewed phpinfo output.
/var/www/html/phpinfo.php
# cat phpinfo.php
<!--?php     phpinfo(); ?-->
    • Secured the MySQL installation, set root to only be allowed from localhost, removed anonymous users, removed the test db.
mysql_secure_installation
    • Downloaded the phpipam tarball.
wget http://downloads.sourceforge.net/project/phpipam/phpipam-1.1.010.tar
    • Connected to MariaDB, created the phpipam DB and granted access to phpipam DB by the ipam user.
# mysql -p
Enter password:
Welcome to the MariaDB monitor. Commands end with ; or \g.
Your MariaDB connection id is 10
Server version: 5.5.41-MariaDB MariaDB Server

Copyright (c) 2000, 2014, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> create database phpipam;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> GRANT ALL on phpipam.* to {IPAM_DB_USER}@localhost identified by '{IPAM_DB_USER_PW}';
Query OK, 0 rows affected (0.00 sec)
    • Modified phpipam/config.php as required.
$db['host'] = "localhost";
$db['user'] = "{IPAM_DB_USER}";
$db['pass'] = "{IPAM_DB_USER_PW}";
$db['name'] = "phpipam";
...
define('BASE', "/phpipam/");
    • Installed the phpipam schema and base DB.
mysql -u root -p phpipam < db/SCHEMA.sql
    • Used browser and connected to the phpipam installation.
http://{SERVER_HOSTNAME}/phpipam/
    • Logged in to phpIPAM and set Admin user password.
    • Missed some PHP settings, was getting the following warnings which were messing up the formatting.
 Warning: date(): It is not safe to rely on the system's timezone settings.
Warning: strtotime(): It is not safe to rely on the system's timezone settings.
    • Set the timezone in the /etc/php.ini file.
; Defines the default timezone used by the date functions
; http://php.net/date.timezone
date.timezone = America/Indianapolis
    • Confirmed I can connect via https.
https://{SERVER_HOSTNAME}/phpipam/
    • Setting up mail settings using our mail relay server and getting the following:
SMTP ERROR: Failed to connect to server: php_network_getaddresses: getaddrinfo failed: Name or service not known
    • Resolves fine from CLI using dig or ping.
    • Some research points to the following bug.

http://sourceforge.net/p/phpipam/bugs/267/
http://sourceforge.net/p/phpipam/patches/_discuss/thread/50111f9a/d932/attachment/phpipam-mail.patch

    • Applied the referenced patch (manually) which fixed that issue.
    • Encountered another issue.
SMTP ERROR: Failed to connect to server: Permission denied (13)
    • Worked fine from cli.
# telnet {RELAY_HOSTNAME} 25
Trying *******...
Connected to {RELAY_HOSTNAME}.
Escape character is '^]'.
220 *****************************************************************************************************************
ehlo
250-BN1AFFO11OLC004.************* Hello [*.*.*.*]
250-SIZE 157286400
250-PIPELINING
250-DSN
250-ENHANCEDSTATUSCODES
250-XXXXXXXA
250-8BITMIME
250-BINARYMIME
250 XXXXXXXB
quit
221 2.0.0 Service closing transmission channel
Connection closed by foreign host.
    • Hmm, might be SELinux or something with PHP? Suspect selinux
    • Checked selinux status.
# sestatus -v
SELinux status: enabled
SELinuxfs mount: /sys/fs/selinux
SELinux root directory: /etc/selinux
Loaded policy name: targeted
Current mode: enforcing
Mode from config file: enforcing
Policy MLS status: enabled
Policy deny_unknown status: allowed
Max kernel policy version: 28

Process contexts:
Current context: unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
Init context: system_u:system_r:init_t:s0
/usr/sbin/sshd unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023

File contexts:
Controlling terminal: unconfined_u:object_r:user_devpts_t:s0
/etc/passwd system_u:object_r:passwd_file_t:s0
/etc/shadow system_u:object_r:shadow_t:s0
/bin/bash system_u:object_r:shell_exec_t:s0
/bin/login system_u:object_r:login_exec_t:s0
/bin/sh system_u:object_r:bin_t:s0 -> system_u:object_r:shell_exec_t:s0
/sbin/agetty system_u:object_r:getty_exec_t:s0
/sbin/init system_u:object_r:bin_t:s0 -> system_u:object_r:init_exec_t:s0
/usr/sbin/sshd system_u:object_r:sshd_exec_t:s0
    • Set the following to address the continuing email issue from within phpIPAM.
# setsebool httpd_can_network_connect=1
    • Now email via phpIPAM is working.
    • At this point phpIPAM was fully functional, the next steps just clean things up and get system settings working as desired.
    • Modified httpd config to allow all overrides in the /var/www/html hierarchy (so mod-rewrite rules, etc from htaccess would work).
/etc/httpd/conf/httpd.conf
<Directory "/var/www/html">
Options Indexes FollowSymLinks
AllowOverride All
Require all granted
    • Restarted httpd.
    • Using the phpIPAM web interface, set “Prettify links” to yes in Administration/”IPAM settings”/”IPAM settings”.
    • Created .htaccess file in the document root to redirect / to https://{SERVER_HOSTNAME}/phpipam/.
# pwd
/var/www/html
# cat .htaccess
RewriteEngine on
RedirectMatch ^/$ https://{SERVER_HOSTNAME}/phpipam/
  • Confirmed the redirect works.
  • Also confirmed the redirect works for http and forces to https.
  • Modified CSS so the header used Arial/sans-serif instead of a serif font.
  • /var/www/html/phpipam/css/bootstrap/bootstrap-custom.css
    .hero-unit a {
    /* font-family: Georgia,Times,'Times New Roman',serif;*/
    font-family: Arial,Helvetica,sans-serif;
    font-size: 36px;color: white;
    text-shadow: black 2px 2px 2px;
    /* white-space: nowrap; */
    }
  • Removed the phpinfo.php file.
  • Next I worked on setting up AD authentication, which was shockingly and refreshingly easy.
  • I created an IPAM group in AD.
  • Placed my domain test user in that group.
  • Noted base DN, this would be different depending on your installation hence the braced {} references. This can be found if you look at the properties of the CN.
  • CN=IPAM,OU={OU_REFERENCE},DC={DOMAIN},DC={TLD}
  • A screenshot of the AD connection settings screen is shown below.
  • phpIPAM AD settings

  • In phpIPAM added my test account as a domain user with an Administrator role. Note that users are still defined in the phpIPAM system (and placed in the desired group), AD is simply used to authenticate the user.
  • At this point I was able to login with my test account and phpIPAM indicated successful AD authentication.
  • In phpIPAM added another test user as a domain user with a “Normal User” role.
  • Confirmed the other test user could log in and only showed the selected hierarchy perms.
  • Created a read-only service account in AD specifically for phpIPAM.
  • One apparent drawback of the “Normal User” role is that even with R/W/A permissions they can’t add new nested subnets. Not a deal breaker, but it means our site admins must provide network hierarchy to the top-level admins to define.
  • The phpIPAM search functionality rocks.
  • We were still on IPPlan 4.92 which won’t make for an easy import.
  • Migrated the old IPPlan 4.92 data to IPPlan to the most recent version Beta6 release (on a different system) which will slightly ease the data migration to phpIPAM.
  • Installed and used phpMyAdmin to export the IPPlan version 6 data to csv.
  • Spent a couple days building the structure I wanted in phpIPAM and importing the host data network by network over the next few days.
  • Next I configured BACKUPS!!!!!
  • Configured Veeam to backup the IPAM server VM every night.
  • Created bash script files for the database backup jobs.
  • # ls -l /usr/local/bin
    total 8
    -rw-r--r-- 1 root root 166 Jun 23 13:51 phpipambackup.sh
    -rw-r--r-- 1 root root 129 Jun 23 13:52 phpipambuclean.sh# cat /usr/local/bin/*
    #!/bin/bash
    # Backup the phpIPAM MariaDB database
    /usr/bin/mysqldump -u {IPAM_DB_USER} -p{IPAM_DB_USER_PW} phpipam > /var/www/html/phpipam/db/bkp/phpipam_bkp_$(date +"%Y%m%d").sql
    
    #!/bin/bash
    # Clean up phpIPAM backup files, keep 21 days
    /usr/bin/find /var/www/html/phpipam/db/bkp/ -ctime +21 -exec rm {} \;
  • Confirmed scripts worked run directly and via crond.
  • # ls -l /var/www/html/phpipam/db/bkp/
    total 3328
    -rw-r--r--. 1 root root 37744 Jun 19 09:38 phpipam_bkp_20150619.sql
    -rw-r--r-- 1 root root 1679688 Jun 23 15:00 phpipam_bkp_20150623.sql
    -rw-r--r-- 1 root root 1679688 Jun 23 14:55 phpipam_bkp_.sql
  • Configured backup to run nightly along with cleanup before the Veeam VM system backup.
  • # cat /etc/crontab
    SHELL=/bin/bash
    PATH=/sbin:/bin:/usr/sbin:/usr/bin
    MAILTO=root
    # For details see man 4 crontabs
    # Example of job definition:
    # .---------------- minute (0 - 59)
    # | .------------- hour (0 - 23)
    # | | .---------- day of month (1 - 31)
    # | | | .------- month (1 - 12) OR jan,feb,mar,apr ...
    # | | | | .---- day of week (0 - 6) (Sunday=0 or 7) OR sun,mon,tue,wed,thu,fri,sat
    # | | | | |
    # * * * * * user-name command to be executed
    # Backup the phpipam database
    01 21 * * * root /usr/local/bin/phpipambackup.sh
    # Clean up the phpipam database backups
    30 21 * * * root /usr/local/bin/phpipambuclean.sh
  • Restarted crond.
  • systemctl restart crond
  • Configured the ipam server to send syslogs to a central syslog server (watches for and reports failed logins, etc.).
  • Confirmed syslogs and reporting working by purposely failing some login in attempts and confirming receipt of syslog server failed login report.

That completed the installation. Importing the data came next, maybe I’ll write about that process in another post.