[MUSIC] Welcome back. In this video, we're going to look at the role Python plays in data science and analytics. Python is an open source, general purpose programming language that can be used for everything from building web applications and enterprise programs to performing analysis on large amounts of data. Python is popular because it is freely available to use, emphasizes code readability, and it is easy for newcomers to learn. Also, it is popular because of its user friendliness and the ability to integrate with a variety of programs, tools and websites. Python has become one of the most popular languages for data management and analysis. Today, Python is widely used by startups and tech companies to embed analytics into their products, and by data scientists to quickly manage and analyze large amounts of data. Python has a set of tools called the Python Data Analytics Stack that address every step of the analytics workflow. These tools are assembled into Python libraries, which are collections of code that are easy to use. While the names of these specific libraries may change, here's a few examples of common libraries for data science. Pandas for importing and assessing data including outline analysis, and data cleansing, as well as summary statistics. NumPY and SciPy for performing extremely fast matrix, mathematical and scientific operations. Statsmodels for fitting a wide range of statistical models to the data. Scikit-Learn from applying machine learning techniques like, clustering, dimensionality reduction, random forests and logistic regression. Matplotlib, Seaborn and Bokeh for producing attractive visuals. And finally, Apache Spark, for processing data on a massive scale across a cluster of computers. In addition to normal data science and numeric data, Python has libraries that can also handle unstructured data like text and images. For example, NLTK, Spacy, and Gensim process text data. OpenCV manipulates and analyzes images. BeautifulSoup and Scrapy make web scraping easy and intuitive. Python can interact with tools like Caffe to use deep learning techniques on powerful GPU enabled machines for cutting edge machine learning on images, sound, and text. And Flask or Django for building websites and web services that can embed machine learning models that can be accessed through the Internet. We've outlined some of the capabilities of Python, but the big question is, why Python? Well, our boasts more extensive, advanced, statistical model and capabilities. Where Python really shines is when the data is unstructured or analytics need to be embedded in other programs or applications. Because Python is a general purpose programming language, it was easy for the community to build extremely efficient libraries to process all kinds of data, and interact with many of the leading big data tools like Hadoop or Apache Spark. Here at PWC, we rely on Python extensively for many of our demonstration projects. We evaluate which leading data tool to use based on the client's need. I'll highlight a couple of examples where Python was used. First, there are projects that use natural language processing. For example, you've heard describe our work to help a major banking client understand the feedback it was getting from it's customers through customer support channels, as well as social media. We use Python to scrape the data, parse and understand it, and then describe the customer's sentiment present in it to better priorities the issues the bank needed to address to improve customer satisfaction. Ultimately, the client was able to better mean customers need and increase customer engagement. On another recent project with an oil and gas client who had a massive amount of census on pumps and oil wells, almost 1 billion records, we used Python as well. The goal was to create a machine learning model to predict when any given pump might fail given its sensor readings. We used Python and Apache Spark to efficiently process the data in parallel so we could do all of our data cleaning and feature engineering before finally fitting a variety of models to it. This was a good example of why Python was chosen, not only because it let us process all high volume data, but also because the client used Python and wanted deployable code that could be kept after the engagement was completed. Ultimately, we were able to provide them with models to predict pump failures two days in advance of an actual failure. Enough time for them to perform maintenance and save on costly pump replacements. Sometimes the choice of the appropriate analysis tool can seem daunting as there are so many options. Python is a great choice because it handles many of the data science use cases, from simple descriptive statistic,s to statistical models, to complex machine learning and distributed computing embedded in dynamic web applications. Python is easy to use because it was designed to be easy for programmers so that the learning curve is not too steep. If you haven't tried it yet, it's worth exploring. In this video, we've discussed the role of Python in data and analytics. Python is free, easy to use, fast, and useful for both simple and complex data analysis tasks. [MUSIC]
