Data Science Training

Data Science

Data Science is an interdisciplinary approach to extracting information from structured and unstructured data. Various techniques, systems, algorithms, and scientific methodologies extract significant insights from random data.

This field focuses on changing data for analysis and visualizations while also utilizing artificial intelligence to provide predictions. Data science requires artificial intelligence.

Simply, Data Science is about data gathering, analysis, decision-making and finding patterns in data through analysis and make future predictions

By using Data Science, companies are able to make:

• Better decisions (should we choose A or B)
• Predictive analysis (what will happen next?)
• Pattern discoveries (find pattern, or maybe hidden information in the data)

 Where is Data Science Needed?

Data science is a versatile field that can be applied to various industries and domains. Here are some areas where data science is needed:

1. Healthcare: Analyzing medical data, patient outcomes, and treatment efficacy.
2. Finance: Risk management, portfolio optimization, and predicting market trends.
3. Marketing: Customer segmentation, personalized recommendations, and campaign analysis.
4. E-commerce: Product recommendations, demand forecasting, and supply chain optimization.
5. Social Media: User behaviour analysis, sentiment analysis, and influencer identification.
6. Sports: Player performance analysis, game prediction, and team strategy optimization.
7. Environmental Science: Climate modeling, pollution analysis, and wildlife conservation.
8. Government: Policy analysis, public health surveillance, and economic development.
9. Education: Student performance analysis, personalized learning, and educational resource optimization.
10. Transportation: Route optimization, traffic prediction, and autonomous vehicles.
11. Energy: Energy consumption forecasting, grid optimization, and renewable energy integration.
12. Manufacturing: Quality control, predictive maintenance, and production optimization.
13. Retail: Inventory management, customer behavior analysis, and store location optimization.
14. Telecommunications: Network optimization, customer churn prediction, and service quality improvement.
15. Research: Scientific data analysis, hypothesis testing, and academic research.

Who is Data Scientists?

Data Scientists are in high demand because we are in data driven world.  Data Scientists are a new breed of professionals who are currently in high demand.

Data Scientist is to extract, analyse and interpret large amounts of data from a range of sources, using algorithmic, data mining, artificial intelligence, machine learning and statistical tools, to make it accessible to businesses.

How Does a Data Scientist Work?

A Data Scientist requires expertise in several backgrounds:

• Machine Learning
• Statistics
• Programming (Python or R)
• Mathematics
• Databases

A Data Scientist must find patterns within the data. Before he/she can find the patterns, he/she must organize the data in a standard format.

Here is how a Data Scientist works:

1. Ask the right questions – To understand the business problem.
2. Explore and collect data – From database, web logs, customer feedback, etc.
3. Extract the data – Transform the data to a standardized format.
4. Clean the data – Remove erroneous values from the data.
5. Find and replace missing values – Check for missing values and replace them with a suitable value (e.g. an average value).
6. Normalize data – Scale the values in a practical range (e.g. 140 cm is smaller than 1.8 m. However, the number 140 is larger than 1.8. – so scaling is important).
7. Analyze data, find patterns and make future predictions.
8. Represent the result – Present the result with useful insights in a way the “company” can understand.

Data Science with Python

Module 1: Introduction to Python

• What is Python?
• Why Python?
• Installing Python
• Python IDEs
• Jupyter Notebook Overview

Module 2: Python Basics

• Python Basic Data types
• Lists
• Slicing
• IF statements
• Loops
• Dictionaries
• Tuples
• Functions
• Array
• Selection by position & Labels

Module 3: Python Packages

• Pandas
• Numpy
• Sci-kit Learn
• Mat-plot library

Module 4: Importing Data

• Reading CSV files
• Saving in Python data
• Loading Python data objects
• Writing data to CSV file

Module 5: Manipulating Data

• Selecting rows/observations
• Rounding Number
• Selecting columns/fields
• Merging data
• Data aggregation
• Data munging techniques

Module 6: Statistics Basics

• Central Tendency
  • Mean
  • Median
  • Mode
  • Skewness
  • Normal Distribution
• Probability Basics
  • What does it mean by probability?
  • Types of Probability
  • ODDS Ratio?

• Standard Deviation
  • Data deviation & distribution
  • Variance

• Bias variance Tradeoff
  • Underfitting
  • Overfitting
• Distance metrics
  • Euclidean Distance
  • Manhattan Distance
• Outlier analysis
  • What is an Outlier?
  • Inter Quartile Range
  • Box & whisker plot
  • Upper Whisker
  • Lower Whisker
  • Scatter plot
  • Cook’s Distance
• Missing Value treatment
  • What is NA?
  • Central Imputation
  • KNN imputation
  • Dummification
• Correlation
  • Pearson correlation
  • positive & Negative correlation

Module 7: Error Metrics

• Classification
  • Confusion Matrix
  • Precision
  • Recall
  • Specificity
  • F1 Score
• Regression
  • MSE
  • RMSE
  • MAPE

Machine Learning

Module 1: Supervised Learning

• Linear Regression
  • Linear Equation
  • Slope
  • Intercept
  • R square value
• Logistic regression
  • ODDS ratio
  • Probability of success
  • Probability of failure Bias Variance Tradeoff
  • ROC curve
  • Bias Variance Tradeoff

Module 2: Unsupervised Learning

• K-Means
• K-Means ++
• Hierarchical Clustering

Module 3: SVM

• Support Vectors
• Hyperplanes
• 2-D Case
• Linear Hyperplane

Module 4: SVM Kernel

• Linear
• Radial
• polynomial

Module 5: Other Machine Learning Algorithms

• K – Nearest Neighbour
• Naïve Bayes Classifier
• Decision Tree – CART
• Decision Tree – C50
• Random Forest

Artificial Intelligence

Module 1: AI Introduction

• Perceptron
• Multi-Layer perceptron
• Markov Decision Process
• Logical Agent & First Order Logic
• AL Applications

Deep Learning

Module 1: Deep Learning Algorithms

• CNN – Convolutional Neural Network
• RNN – Recurrent Neural Network
• ANN – Artificial Neural Network

Module 2: Introduction to NLP

• Text Pre-processing
• Noise Removal
• Lexicon Normalization
• Lemmatization
• Stemming
• Object Standardization

Module 3: Text to Features (Feature Engineering)

• Syntactical Parsing
• Dependency Grammar
• Part of Speech Tagging
• Entity Parsing
• Named Entity Recognition
• Topic Modelling
• N-Grams
• TF – IDF
• Frequency / Density Features
• Word Embedding’s

Module 4: Tasks of NLP

• Text Classification
• Text Matching
• Levenshtein Distance
• Phonetic Matching
• Flexible String Matching

Tableau

Module 1: Tableau Course Material

• Start Page
• Show Me
• Connecting to Excel Files
• Connecting to Text Files
• Connect to Microsoft SQL Server
• Connecting to Microsoft Analysis Services
• Creating and Removing Hierarchies
• Bins
• Joining Tables
• Data Blending

Module 2: Learn Tableau Basic Reports

• Parameters
• Grouping Example 1
• Grouping Example 2
• Edit Groups
• Set
• Combined Sets
• Creating a First Report
• Data Labels
• Create Folders
• Sorting Data
• Add Totals, Subtotals and Grand Totals to Report

Module 4: Learn Tableau Advanced Reports

• Dual Axis Reports
• Blended Axis
• Individual Axis
• Add Reference Lines
• Reference Bands
• Reference Distributions
• Basic Maps
• Symbol Map
• Use Google Maps
• Mapbox Maps as a Background Map
• WMS Server Map as a Background Map

Module 5: Learn Tableau Calculations & Filters

• Calculated Fields
• Basic Approach to Calculate Rank
• Advanced Approach to Calculate Ra
• Calculating Running Total
• Filters Introduction
• Quick Filters
• Filters on Dimensions
• Conditional Filters
• Top and Bottom Filters
• Filters on Measures
• Context Filters
• Slicing Filters
• Data Source Filters
• Extract Filters

Module 6: Learn Tableau Dashboards

• Create a Dashboard
• Format Dashboard Layout
• Create a Device Preview of a Dashboard
• Create Filters on Dashboard
• Dashboard Objects
• Create a Story

Module 7: Server

• Tableau online.
• Overview of Tableau
• Publishing Tableau objects and scheduling/subscription.

SQL

Module 1: Introduction to Database

• List the features of Oracle Database 11g
• Discuss the basic design, theoretical, and physical aspects of a relational database
• Categorize the different types of SQL statements
• Describe the data set used by the course
• Log on to the database using SQL Developer environment
• Save queries to files and use script files in SQL Developer

Module 2: Retrieve Data using the SQL SELECT Statement

• List the capabilities of SQL SELECT statements
• Generate a report of data from the output of a basic SELECT statement
• Select All Columns
• Select Specific Columns
• Use Column Heading Defaults
• Use Arithmetic Operators
• Understand Operator Precedence
• Learn the DESCRIBE command to display the table structure

Module 3: Learn to Restrict and Sort Data

• Write queries that contain a WHERE clause to limit the output retrieved
• List the comparison operators and logical operators that are used in a WHERE clause
• Describe the rules of precedence for comparison and logical operators
• Use character string literals in the WHERE clause
• Write queries that contain an ORDER BY clause to sort the output of a SELECT statement
• Sort output in descending and ascending order

Module 4: Usage of Single-Row Functions to Customize Output

• Describe the differences between single row and multiple row functions
• Manipulate strings with character function in the SELECT and WHERE clauses
• Manipulate numbers with the ROUND, TRUNC, and MOD functions
• Perform arithmetic with date data
• Manipulate dates with the DATE functions

Module 5: Invoke Conversion Functions and Conditional Expressions

• Describe implicit and explicit data type conversion
• Use the TO_CHAR, TO_NUMBER, and TO_DATE conversion functions
• Nest multiple functions
• Apply the NVL, NULLIF, and COALESCE functions to data
• Use conditional IF THEN ELSE logic in a SELECT

Module 6: Aggregate Data Using the Group Functions

• Use the aggregation functions in SELECT statements to produce meaningful reports
• Divide the data into groups by using the GROUP BY clause
• Exclude groups of date by using the HAVING clause

Module 7: Display Data from Multiple Tables Using Joins

• Write SELECT statements to access data from more than one table
• View data that generally does not meet a join condition by using outer joins
• Join a table by using a self-join

Module 8: Use Subqueries to Solve Queries

• Describe the types of problem that subqueries can solve
• Define sub-queries
• List the types of sub-queries

Module 9: The SET Operators

• Describe the SET operators
• Use a SET operator to combine multiple queries into a single query
• Control the order of rows returned

Module 10: Data Manipulation Statements

• Describe each DML statement
• Insert rows into a table
• Change rows in a table by the UPDATE statement
• Delete rows from a table with the DELETE statement
• Save and discard changes with the COMMIT and ROLLBACK statements
• Explain read consistency

Module 11: Use of DDL Statements to Create and Manage Tables

• Categorize the main database objects
• Review the table structure
• List the data types available for columns
• Create a simple table
• Decipher how constraints can be created at table creation
• Describe how schema objects work

Module 12: Other Schema Objects

• Create a simple and complex view
• Retrieve data from views
• Create, maintain, and use sequences
• Create and maintain indexes
• Create private and public synonyms

Module 13: Control User Access

• Differentiate system privileges from object privileges
• Create Users
• Grant System Privileges
• Create and Grant Privileges to a Role
• Change Your Password
• Grant Object Privileges
• How to pass on privileges?
• Revoke Object Privileges

Module 14: Management of Schema Objects

• Add, Modify and Drop a Column
• Add, Drop and Defer a Constraint
• How to enable and Disable a Constraint?
• Create and Remove Indexes
• Create a Function-Based Index
• Perform Flashback Operations
• Create an External Table by Using ORACLE_LOADER and by Using ORACLE_DATAPUMP
• Query External Tables

Module 15: Manage Objects with Data Dictionary Views

• Explain the data dictionary
• Use the Dictionary Views
• USER_OBJECTS and ALL_OBJECTS Views
• Table and Column Information
• Query the dictionary views for constraint information
• Query the dictionary views for view, sequence, index, and synonym information
• Add a comment to a table
• Query the dictionary views for comment information

Module 16: Manipulate Large Data Sets

• Use Subqueries to Manipulate Data
• Retrieve Data Using a Subquery as Source
• Insert Using a Subquery as a Target
• Usage of the WITH CHECK OPTION Keyword on DML Statements
• List the types of Multitable INSERT Statements
• Use Multitable INSERT Statements
• Merge rows in a table
• Track Changes in Data over a period of time

Module 17: Data Management in Different Time Zones

• Time Zones
• CURRENT_DATE, CURRENT_TIMESTAMP, and LOCALTIMESTAMP
• Compare Date and Time in a Session’s Time Zone
• DBTIMEZONE and SESSIONTIMEZONE
• Difference between DATE and TIMESTAMP
• INTERVAL Data Types
• Use EXTRACT, TZ_OFFSET, and FROM_TZ
• Invoke TO_TIMESTAMP, TO_YMINTERVAL and TO_DSINTERVAL

Module 18: Retrieve Data Using Sub-queries

• Multiple-Column Subqueries
• Pairwise and Non Pairwise Comparison
• Scalar Subquery Expressions
• Solve problems with Correlated Subqueries
• Update and Delete Rows Using Correlated Subqueries
• The EXISTS and NOT EXISTS operators
• Invoke the WITH clause
• The Recursive WITH clause

Module 19: Regular Expression Support

• Use the Regular Expressions Functions and Conditions in SQL
• Use Meta Characters with Regular Expressions
• Perform a Basic Search using the REGEXP_LIKE function
• Find patterns using the REGEXP_INSTR function
• Extract Substrings using the REGEXP_SUBSTR function
• Replace Patterns Using the REGEXP_REPLACE function
• Usage of Sub-Expressions with Regular Expression Support
• Implement the REGEXP_COUNT function