Monthly Archives: June 2017

GSOC: Week 5

This weekend I focused on finishing the first part of the project (eeg data analysis pipeline) and making the user facing GUI. Regarding the GUI, this week was mainly for planning and I managed to create a skeleton architecture. During … Continue reading

Posted in Google summer of code | Leave a comment

GSOC: Week 4

This weekend I focused on implementing the full machine learning pipeline which means we can do everything from loading the EEG up-to giving the predicted responses. What I did this week: Created the Classifier Interface Implemented the Logistic Regression classifier … Continue reading

Posted in Google summer of code | Leave a comment

Get Spark Clasifier metrics using the Confusion Matrix

When using Apache Spark specifically for “binary” classification (ie. the labels are either 0 or 1) it is possible to use the Confusion Matrix for getting some metrics such as the number of true positives, false negatives or true negatives. … Continue reading

Posted in Data Engineering, Data science | Leave a comment

GSOC: Week 3

I had a very productive week managing to implement the feature engineering process using Discrete Wavelet transforms. I also made the according and will start working on classification procedure. What I did this week: Create modular interface for signal processing … Continue reading

Posted in Google summer of code | Leave a comment

How to fix ‘Task not serializable’ issues in Apache Spark

When using the RDD API, you can write Map functions which can serve as complex closures.  Because each Map function is executed in parallel on one of the executors, the functionality inside the Map phase, (ie. the code) is sent … Continue reading

Posted in Data Engineering, Data science | Leave a comment

GSOC: Week 2

I was relatively inactive this week as I had my final exams for this academic year on Tuesday, Thursday and Friday. I mainly focused on planning the future course of the project. What I did this week: Improved project management by … Continue reading

Posted in Google summer of code | Leave a comment

GSOC: Week 1

I worked on improving the code to match some quality standards my mentor advised. Mainly I added logging support instead of pure printing as well as changed the project structure. What I did this week: Replaced print statements with logging … Continue reading

Posted in Google summer of code | Leave a comment