Exploring Distributed Systems, Machine Learning and Operations, and Engineering Management
Yo, fellow Jupyter notebook user! Have you ever wished for some extra features to make your notebook experience even better? Well, have no fear, because "notebook extensions" are here! These handy little plug-ins can easily be added to your Jupyter notebooks to enhance their functionality. To get started, we’ll need to install the "Jupyter NbExtensions […]
So you’ve been using Jupyter notebooks for a while now and the default light theme is starting to make you feel like a vampire. Fear not, my friend! I have just the solution for you. Enter the "jupyter-themes" library – a free and open-source tool that lets you install and customize themes to your heart’s […]
Linear Regression is the most commonly used predictive analysis. It is used to model the relationship between a dependent variable and one or more independent variables. In this project we create an ML pipeline to train a linear regression model to predict the release year of a song given a set of audio features. We […]
Hey there! So we decided to create a Word Count application – a classic MapReduce example. But what the heck is a Word Count application, you ask? It’s basically a program that reads data and calculates the most common words. Easy peasy. For example: dataDF = sqlContext.createDataFrame([('Jax',), ('Rammus',), ('Zac',), ('Xin', ), ('Hecarim', ), ('Zac', ), […]
Hey, so I’m a big fan of Apache Spark and I’ve been using it for all of my independent projects. I recently had this idea to create a project that would showcase how to do some data wrangling with Apache Spark. For this project, we used Apache Spark 2.0.2 on Databricks cloud. Instead of using […]