Overview
Once you’ve identified a big data issue to analyze, how do you collect, store and organize your data using Big Data solutions? In this course, you will experience various data genres and management tools appropriate for each. You will be able to describe the reasons behind the evolving plethora of new big data platforms from the perspective of big data management systems and analytical tools. Through guided hands-on tutorials, you will become familiar with techniques using real-time and semi-structured data examples. Systems and tools discussed include: AsterixDB, HP Vertica, Impala, Neo4j, Redis, SparkSQL. This course provides techniques to extract value from existing untapped data sources and discovering new data sources.
At the end of this course, you will be able to:
* Recognize different data elements in your own work and in everyday life problems
* Explain why your team needs to design a Big Data Infrastructure Plan and Information System Design
* Identify the frequent data operations required for various types of data
* Select a data model to suit the characteristics of your data
* Apply techniques to handle streaming data
* Differentiate between a traditional Database Management System and a Big Data Management System
* Appreciate why there are so many data management systems
* Design a big data information system for an online game company
This course is for those new to data science. Completion of Intro to Big Data is recommended. No prior programming experience is needed, although the ability to install applications and utilize a virtual machine is necessary to complete the hands-on assignments. Refer to the specialization technical requirements for complete hardware and software specifications.
Hardware Requirements:
(A) Quad Core Processor (VT-x or AMD-V support recommended), 64-bit; (B) 8 GB RAM; (C) 20 GB disk free. How to find your hardware information: (Windows): Open System by clicking the Start button, right-clicking Computer, and then clicking Properties; (Mac): Open Overview by clicking on the Apple menu and clicking “About This Mac.” Most computers with 8 GB RAM purchased in the last 3 years will meet the minimum requirements.You will need a high speed internet connection because you will be downloading files up to 4 Gb in size.
Software Requirements: This course relies on several open-source software tools, including Apache Hadoop. All required software can be downloaded and installed free of charge (except for data charges from your internet provider). Software requirements include: Windows 7+, Mac OS X 10.10+, Ubuntu 14.04+ or CentOS 6+ VirtualBox 5+.
Syllabus
- Introduction to Big Data Modeling and Management
- Welcome to this course on big data modeling and management. Modeling and managing data is a central focus of all big data projects. In these lessons we introduce you to the concepts behind big data modeling and management and set the stage for the remainder of the course.
- Big Data Modeling
- Modeling big data depends on many factors including data structure, which operations may be performed on the data, and what constraints are placed on the models. In these lessons you will learn the details about big data modeling and you will gain the practical skills you will need for modeling your own big data projects.
- Big Data Modeling (Part 2)
- These lessons continue to shed light on big data modeling with specific approaches including vector space models, graph data models, and more.
- Working With Data Models
- Data models deal with many different types of data formats. Streaming data is becoming ubiquitous, and working with streaming data requires a different approach from working with static data. In these lessons you will gain practical hands-on experience working with different forms of streaming data including weather data and twitter feeds.
- Big Data Management: The “M” in DBMS
- Managing big data requires a different approach to database management systems because of the wide variation in data structure which does not lend itself to traditional DBMSs. There are many applications available to help with big data management. In these lessons we introduce you to some of these applications and provide insight into how and when they might be appropriate for your own big data management challenges.
- Designing a Big Data Management System for an Online Game
- In these lessons we give you the opportunity to learn about big data modeling and management using a fictitious online game called “Catch the Pink Flamingo”.