Software DevelopersPrepare for Your next Job Opportunities

I am amazed at how the greater public perception of software functionality is still focused on web applications. While web application developers are probably filling the bulk of IT-related job market at this time, this job market segment has become saturated and is gradually trending down in terms of both demand and job compensation expectations.

Throughout my many years as a computer technology consultant I learned that the best income is derived from opportunities that are present in niche markets. The main challenge is to study the market and identify such niche opportunities, because many times even your customers do not know what to ask for. Great advantages are waiting for those who prepare for the future and are ready to serve niche requirements before others discover them and before they become main-stream and later on, turn into commodity.

During my years of service at Intel Corp. as a Systems Software Architect, I saw numerous opportunities for company operations to directly benefit from Machine Learning applications. While serving on the staff of the Electrical Validation department, I actually initiated two company wide, strategic projects that could have directly saved the company 30-40 million dollars in labor hours each year and in addition, cut critical response time to chip manufacturing issues, from weeks, to days or even hours. The latter capability in particular, was critical in my mind, to retain the company in a competitive position, while expanding the customer base that it serves, into smaller OEMs. Unfortunately, corporate Intel initiated the mindless layoffs of 2015, which caught me in their net and consequently, Intel Corp. will never see the cost savings and efficiency improvements out of my two projects that, due to my absence, met a silent death.

Operations Research (OR)  and Machine Learning (ML) go hand in hand nowadays. Operations Research has roots in in the first half of the 20th century (WW-II war planning) and a long history of applications. Although Arthur Samuel was credited with development of the first computer learning program, playing checkers against a human opponent (in 1956), Machine Learning remained mostly on the books and only began evolving in a serious way over the last two decades of our time.

Arthur Samuel and the IBM 701 playing checkers in 1956

Machine learning is a technology that its time has arrived. Though you can run machine learning code on any average computer, most automatic learning algorithms benefit from execution on large scale hardware once you begin dealing with very large data-sets. Nowadays, you do not need to actually own expensive hardware, because highly scale-able and affordable computing apparatus is accessible à la carte (pay per use) from companies like Google, Amazon, and other operators. Anyone willing to apply machine learning to solve practical problems can get access to advanced computing capabilities, without much up-front capital investment.

To quote Wikipedia, modern Machine Learning classifies three broad categories of applications:

• Supervised learning: The computer is presented with example inputs and their desired outputs, given by a "teacher", and the goal is to learn a general rule that maps inputs to outputs.
• Unsupervised learning: No labels are given to the learning algorithm, leaving it on its own to find structure in its input. Unsupervised learning can be a goal in itself (discovering hidden patterns in data) or a means towards an end (feature learning).
• Reinforcement learning: A computer program interacts with a dynamic environment in which it must perform a certain goal (such as driving a vehicle), without a teacher explicitly telling it whether it has come close to its goal. Another example is learning to play a game by playing against an opponent.

For those who are not afraid to learn and apply their learning, I highly recommend looking at sharpening your skills in the art of machine learning programming.

The Journal of Machine Learning Research (JMLR) provides a large collection of research papers, articles, and software, covering the gamut of ML algorithms, frameworks, and applications. Click on the link below to navigate into the main software page of this site:
     JMLR Software Page

Good summary of the ML field can be found in this article:

     The 10 Algorithms Machine Learning Engineers Need to Know

Another article on the same web site includes some examples of code written to take advantage of the Google sponsored TensorFlow  machine learning framework to apply ML solutions for data analysis.

To read this article click here: Tensorflow -- 3 Ways

Note that Google actually developed a specialized and power efficient ML hardware architecture (called the TesnsorFlow Machine), utilizing a large array of Tensor Processing Units (TPU). This architecture provides several orders of magnitude improvement in power/performance ratio for ML applications.

Read the full article about the TensorFlow Machine by clicking on this link:
Google's Tensor Processing Unit explained: this is what the future of computing looks like

TensorFlow tutorials are available here: Tutorials

This is another machine learning framework that is available in Open Source form, through the Apache organization. PredictionIO website:
https://predictionio.incubator.apache.org/index.html contains a lot of introductory information, including demos and tutorials.

For those of you who are versed in Python I recommend looking at:

scikit-learn

Scikit-learn is built on top of the very common NumPy, SciPy, and matplotlib libraries. Scikit-learn provides an easy to use python-based interface to a large variety of ML algorithms and functions and allows pythons data analysis programs to process and display the resulting data in a python native way.

The Machine Learning chapter of GitHub provides a directory of ML open source projects. Some of these have been mentioned in this article, while the rest are left for you to explore on your own.

The above mentioned ML frameworks are only selective examples of the growing number of ML tools in existence today. I suggest to choose one of them to start with, to get your feet wet. Once you get proficient in understanding of what ML can do for your application, you can move on to another framework and compare their capabilities.

Your comments are welcome. Please click on the commenting link below the article and submit your input.

--Dr. Flywheel