| Home . Portfolio . PHP Framework . Development Blog . Contact | Login |
|
My core competency is in the design, development and deployment of web applications. The below applications are samples of some of the projects I have been involved in. Data Set Validator - NASA Planetary Data System
Fig. 1 - Applet Validation While open to the public, this validation tool (see about page) will be of little use if you do not have data in the Planetary Data System archive format (PDS3). Some background on the tool is necessary to evaluate the appropriateness and effectiveness of the solution, namely a cursory understanding of the PDS and the archive process. The PDS is responsible for the long term storage of data collected from missions and resurfacing that data to the public. This involves the management of a storage format/process that allows the data to be machine readable, searchable, and be processed hundreds of years into the future. It also involves the evaluation of data for scientific relevance and accuracy. 
Fig. 2 - Validation Results Prior to this tool, it was very difficult to evaluate the hundreds of thousands of files that may be in a single data set for adherence to the PDS3 standard or determine if there were any errors or anomalies. There were a variety of targeted command-line tools that required a great deal of time, patience, and skill to make use of. Furthermore, each of these tools was file-centric, ignoring many of the rules about structure and interdependence of entire data sets. On top of this, there was a great deal of fragmentation in tools, everyone using different versions and/or being unaware of other tools that addressed a different portion of the problem. 
Fig. 3 - Filtered Results My solution was to build a tool that encompassed all of the existing functionality as well as addressing data set wide issues. To combat fragmentation and the support for a variety of operating systems, this tool was built as a web application. Due to the large number and size of files however, the validation process is performed on a user's local machine by means of a java web applet (Fig. 1). Once validation is complete, the results are posted to the server for display (Fig. 2). Filtering (Fig. 3), navigation, and analysis of the results is performed on the server without need of re-validation. You may also preview files with offending lines highlighted (Fig. 4). 
Fig. 4 - File Preview After some initial difficulty with adoption (the community showed a great deal of reluctance for web applications and GUI's in general), the tool has received a great deal of support. Analytics are showing both that assumptions of initial quality of existing data sets were artificially high and that use of this tool has had a significant positive impact on the quality of data being ingested as well as the confidence level that tools based on the current standard can search and manipulate this data. If you would like to demo this tool, a number of data sets are available for public download. However, much of the information and display format is targeted for data preparers, archivers, and node staff. You may find the results somewhat difficult to make sense of without intimate knowledge of the system. With regards to my role on this project, beyond some advice and code snippets from colleagues, the tool was developed in its entirety by myself. This includes:
MySQL Enterprise Monitor
Fig. 5 - Dashboard MySQL Enterprise Monitor is actually a framework for solutions rather than a single targeted application. In it's simplest form, it acts like a virtual DBA, checking for security issues, performance problems, down machines, etc (Fig. 5). More specifically, it does rule base expression evaluation for the purposes of alerting issues and graphing trends. It also visualizes replication topologies (Fig. 7), monitoring their health (Fig. 6). Furthermore, if you flip a switch on the agents that do the monitoring, they can they act as proxies that allow query analysis or even query load balancing. 
Fig. 6 - Heatchart My roll in this project was designer, usability engineer, and front end developer. I was responsible for all layers down to an application API, including designing mock interfaces for business objects and generating failing unit tests to provide developers. Some of the more challenging elements of the project were the complexity of concepts requiring simple visualizations, issues of scale (imagine hundreds of servers flooding you with information) and internationalization for Japanese markets. 
Fig. 7 - Replication As the application was meant to serve as a real time monitor and dashboard, views had to be extremely information rich and always up to date. To this end, a surprising portion of the work was in advanced JavaScript. Samples of this work will be available shortly in the Programming section.
|
| privacy . bio . resume . contact | © 2012 HolisticMonkey Inc. ALL RIGHTS RESERVED |