Machine Learning on

Exploring Reinforcement Learning Concepts: A Comprehensive Guide

hari@dasarpai.com (Dr. Hari Thapliyaal) — Sat, 22 Feb 2025 00:00:00 +0000

Exploring Reinforcement Learning Concepts
#

Reinforcement Learning (RL) is a rich and complex field with many important concepts. Here are some high level concepts which you need to understand, and explore this field.

Key Concepts of Reinforcement Learning (RL)
#

1. Markov Decision Processes (MDPs)
#

Definition: The mathematical framework for RL, consisting of states, actions, transitions, and rewards.
Key Components:
- State (S): The current situation of the agent.
- Action (A): Choices available to the agent.
- Transition Function (P): Probability of moving to a new state given an action.
- Reward Function (R): Immediate feedback for taking an action in a state.
- Discount Factor (γ): Determines the importance of future rewards.
Extensions:
- Partially Observable MDPs (POMDPs): When the agent cannot fully observe the state.
- Continuous MDPs: For continuous state and action spaces.

2. Policies
#

Definition: A strategy that the agent uses to decide actions based on states.
Types:
- Deterministic Policy: Maps states to specific actions.
- Stochastic Policy: Maps states to probability distributions over actions.
Optimal Policy: The policy that maximizes cumulative rewards.

3. Value Functions
#

State-Value Function (V): Expected cumulative reward from a state under a policy.
Action-Value Function (Q): Expected cumulative reward for taking an action in a state and following a policy.
Bellman Equation: Recursive relationship used to compute value functions.

4. Exploration vs. Exploitation
#

Exploration: Trying new actions to discover their effects.
Exploitation: Choosing known actions that yield high rewards.
Balancing Mechanisms:
- ε-Greedy: Randomly explores with probability ε.
- Softmax: Selects actions based on a probability distribution.
- Upper Confidence Bound (UCB): Balances exploration and exploitation based on uncertainty.

5. Algorithms
#

Model-Based vs. Model-Free:
- Model-Based: Learns a model of the environment (transition and reward functions).
- Model-Free: Learns directly from interactions without modeling the environment.
Key Algorithms:
- Q-Learning: Off-policy algorithm for learning action-value functions.
- SARSA: On-policy algorithm for learning action-value functions.
- Deep Q-Networks (DQN): Combines Q-learning with deep neural networks.
- Policy Gradient Methods: Directly optimize the policy (e.g., REINFORCE, PPO, TRPO).
- Actor-Critic Methods: Combines value-based and policy-based approaches.

6. Function Approximation
#

Purpose: Handles large or continuous state/action spaces.
Methods:
- Linear Approximation: Uses linear combinations of features.
- Neural Networks: Deep learning for complex function approximation.
Challenges:
- Overfitting, instability, and divergence.

7. Temporal Difference (TD) Learning
#

Definition: Combines Monte Carlo methods and dynamic programming for online learning.
Key Concepts:
- TD Error: Difference between estimated and actual returns.
- Bootstrapping: Updating estimates based on other estimates.

8. Eligibility Traces
#

Purpose: Improves efficiency of TD learning by considering recent states and actions.
Example: TD(λ), where λ controls the trace decay.

9. Multi-Agent RL (MARL)
#

Definition: Extends RL to environments with multiple agents.
Challenges:
- Non-stationarity (other agents are also learning).
- Coordination and competition.
Approaches:
- Cooperative, Competitive, and Mixed settings.

10. Transfer Learning in RL
#

Definition: Applying knowledge from one task to another.
Methods:
- Domain Adaptation: Adjusting to new environments.
- Skill Transfer: Reusing learned policies or value functions.

11. Safe and Ethical RL
#

Safe Exploration: Avoiding harmful actions during learning.
Ethical Constraints: Incorporating human values into reward design.

12. Hierarchical RL (HRL)
#

Definition: Breaks tasks into sub-tasks or sub-goals.
Methods:
- Options Framework: Temporal abstractions for actions.
- MAXQ: Hierarchical decomposition of value functions.

13. Imitation Learning
#

Definition: Learning from expert demonstrations.
Methods:
- Behavior Cloning: Supervised learning to mimic expert actions.
- Inverse RL: Inferring the reward function from demonstrations.

14. Meta-Learning in RL
#

Definition: Learning to learn, or adapting quickly to new tasks.
Methods:
- Model-Agnostic Meta-Learning (MAML): Adapts to new tasks with few samples.
- RL²: Treats the RL algorithm itself as a learning problem.

15. Exploration Strategies
#

Intrinsic Motivation: Encourages exploration through curiosity or novelty.
Count-Based Exploration: Rewards visiting rare states.
Random Network Distillation (RND): Uses prediction errors to drive exploration.

16. Challenges in RL
#

Sample Efficiency: Learning with limited interactions.
Credit Assignment: Determining which actions led to rewards.
Scalability: Handling high-dimensional state/action spaces.
Stability: Avoiding divergence during training.

17. Applications of RL
#

Games: AlphaGo, Dota 2, Chess.
Robotics: Manipulation, locomotion, autonomous driving.
Healthcare: Personalized treatment, drug discovery.
Finance: Portfolio optimization, trading strategies.
Recommendation Systems: Personalized content delivery.

18. Tools and Frameworks
#

Libraries:
- OpenAI Gym: Standardized environments for RL.
- Stable-Baselines3: Implementations of RL algorithms.
- Ray RLlib: Scalable RL for distributed computing.
Simulators:
- MuJoCo, PyBullet, Unity ML-Agents.

19. Theoretical Foundations
#

Convergence Guarantees: Conditions under which RL algorithms converge.
Regret Minimization: Balancing exploration and exploitation over time.
Policy Improvement Theorems: Guarantees for improving policies iteratively.

20. Advanced Topics
#

Off-Policy Learning: Learning from data generated by a different policy.
Offline RL: Learning from pre-collected datasets without interaction.
Multi-Task RL: Learning multiple tasks simultaneously.
Meta-RL: Learning RL algorithms themselves.

What are differening rewardng systems in RL?
#

In reinforcement learning (RL), reward systems are pivotal in guiding agents to learn optimal behaviors. Here’s an organized overview of different reward systems, their characteristics, and applications:

Exploring Types of Models

hari@dasarpai.com (Dr. Hari Thapliyaal) — Sat, 25 Jan 2025 00:00:00 +0000

Understanding Different Types of Models
#

Introduction
#

A model is a simplified representation or abstraction of a system, concept, or phenomenon around us. It is used to analyze, understand, predict, or simulate real-world behavior. Models can take many forms, depending on the context in which they are used. For example you also say that I have created a Data Model, Functional Model, UI Model, Simulation Model etc.

AI ML Project Ideas

hari@dasarpai.com (Dr. Hari Thapliyaal) — Tue, 21 Jan 2025 00:00:00 +0000

AI Project Ideas
#

Introduction
#

In your journey of learning and practicing AI project you may find these articles and repositories useful. I found it over many places and compiled them here. Some of the link may break. These are beginner level ideas for AI project. If you are starting your journey then you will find value in exploring these. If you have been seriously for more than 5+ years in AI/ML then your can ignore this article.

Transfer Learning Key AI Techniques Explained

hari@dasarpai.com (Dr. Hari Thapliyaal) — Fri, 25 Oct 2024 00:00:00 +0000

Transfer Learning Key AI Techniques Explained
#

In this article we will understand some important concepts used within machine learning.

What is in-context Learning?
What is Prompt-Engineering?
What is the relationship between Prompt Engineering and In-Context Learning?
What is Zero-shot learning?
How Zero-shot learning is different from In-context Learning?
What is Meta-Learning?
What is Few-shot learning?
Do we need foundational models for Meta-learning and Few-shot learning?
What is transfer learning?
How do we do transfer learning from existing model?
What is finetuning?
Which layers to update, what weight to update during finetuning?

Prompt Engineering, In Context Learning and Zero-shot Learning
#

What is in-context Learning?
#

In-Context Learning refers to a model’s ability to adapt its responses based on the context provided in the input prompt without updating its parameters or undergoing explicit training. The model uses the examples, instructions, or context given in the input to influence its behavior during inference.

Machine Learning Key Concepts

hari@dasarpai.com (Dr. Hari Thapliyaal) — Thu, 03 Oct 2024 00:00:00 +0000

Machine Learning Key Concepts
#

In this article Essential Machine Learning Techniques/Concepts are Explained, some of them are are Cross-Validation, Hyperparameter Optimization, Machine learning types and much More.

Is this article for me?
#

If you are looking for the answer to any of the following questions, then the answer is ‘Yes.’

Automated Machine Learning

hari@dasarpai.com (Dr. Hari Thapliyaal) — Tue, 01 Oct 2024 00:00:00 +0000

Automated Machine Learning
#

Is this article for me?
#

This is article is for you, if you know

About Machine learning, ML models building
That machines are capable of building these models themselves.

But you don’t know how it happens and what are different libraries available for this work.

Tensorflow GPU Setup on Local Machine

hari@dasarpai.com (Dr. Hari Thapliyaal) — Wed, 28 Aug 2024 00:00:00 +0000

Tensorflow GPU Setup on Local Machine
#

Introduction
#

Tensorflow, pytorch are deep learning libraries or packages. Tensorflow is developed by google and pytorch is developed by Meta. There are some other but these are the most popular one among Machine Learning and Deep Learning Engineers. If you are doing anything significant in NLP, computer vision, voice processing you must have used this library. But the power of the these libraries lies in parallel metrics/tensor computation. For that they use hardwardes like GPU or TPU which has thousands of core and they designed purely for metrics/tensor processing. Intially they were used for gaming purpose but with the surge of AI these machines are in high use and used for model training and inference purpose.

MLOps Tools

hari@dasarpai.com (Dr. Hari Thapliyaal) — Tue, 13 Aug 2024 00:00:00 +0000

MLOps Tools
#

MLOps (Machine Learning Operations) is a set of practices and tools designed to streamline and automate the deployment, monitoring, and management of machine learning models in production environments. It combines principles from both DevOps (Development Operations) and machine learning to ensure that ML models are deployed efficiently, managed effectively, and maintained reliably throughout their lifecycle.

Why to Finetune LLM?

hari@dasarpai.com (Dr. Hari Thapliyaal) — Sun, 28 Jul 2024 00:00:00 +0000

Finetuning, Fewshot Learning, Why and How?
#

Why to finetune a LLM?
#

Fine-tuning a large language model (LLM) can provide several benefits, depending on your specific needs and objectives. Here are some key reasons to consider fine-tuning an LLM:

Dimensionality Reduction and Visualization

hari@dasarpai.com (Dr. Hari Thapliyaal) — Wed, 24 Jul 2024 00:00:00 +0000

Dimensionality Reduction and Visualization
#

What are the popular methods of dimensionality reduction?
#

Dimensionality reduction is a crucial step in data preprocessing, particularly when dealing with high-dimensional datasets. It helps in reducing the number of features while retaining the essential information, improving computational efficiency, and facilitating data visualization. Here are some popular methods of dimensionality reduction:

ML Model Development Framework

hari@dasarpai.com (Dr. Hari Thapliyaal) — Sat, 02 Sep 2023 00:00:00 +0000

ML Model Development Framework & Model Repositories
#

Introduction
#

There are hundreds of machine learning tasks. To do these tasks there are thousands of datasets created by individuals, governments, and corporations. We need to develop AI models using these datasets. There are thousands of models 1, 2, 3 and many model development frameworks. It is practically mind-blowing to track this whole body of work and understand all this work in its entirety. But if you dive deeper into the following frameworks you will get a fair idea about the overall direction of the work. These frameworks are used to maintain pre-trained model repositories and download pre-trained models. You can develop your own finetuned model using those pre-trained models.

ML Model Respository from Pinto0309

hari@dasarpai.com (Dr. Hari Thapliyaal) — Fri, 01 Sep 2023 00:00:00 +0000

ML Model Repository from Pinto0309
#

Introduction
#

Using AI we can solve many kinds of tasks for this input can be text, structured data, image, video, audio, time-series, etc. To solve these problems we need to train model. These models may be computer vision, NLP, or traditional machine learning kind. There are hundreds of architectures and algorithms to solve business problems and create models. There a hundreds of different datasets that can be along with a particular architecture or algorithm to solve the problem. If you have any of these tasks then you can explore using these pre-trained models to solve your problem. There is a GitHub user “Katsuya Hyodo” with GitHub account “PINTO0309”. He has trained hundreds of models and created these pre-trained models for the community. You can scan and explore them from there. From there you can download the pre-trained models.

Distances in Machine Learning

hari@dasarpai.com (Dr. Hari Thapliyaal) — Sun, 27 Aug 2023 00:00:00 +0000

Distances in Machine Learning
#

Every sample, record, word, sentence, object, image etc in the Machine learning language is called vector. If we want to measure the similarity or dissimilarity between two data points then we need distance function.

Distance metrics play a crucial role in various machine learning algorithms, including clustering, classification, and anomaly detection. Different distance measures capture different types of relationships between data points.

Machine Learning Metrics

hari@dasarpai.com (Dr. Hari Thapliyaal) — Mon, 21 Aug 2023 00:00:00 +0000

Machine Learning Metrics
#

Introduction
#

In Machine Learning projects whether classical machine learning, deep learning, computer vision, speech processing, NLP, or any other ML project we keep building different models with different datasets. But how to know that for a particular problem model X is the best one? For that, we need to evaluate these models against certain metrics. What metrics we pick, depends upon the problem statement, data imbalance, type of data, etc. In this article, we will explore an exhaustive list of ML Metrics.

Introduction to ML Model Deployment

hari@dasarpai.com (Dr. Hari Thapliyaal) — Wed, 19 Jul 2023 00:00:00 +0000

Introduction to AI Model deployment
#

Big Players
#

Amazon
- Amazon has many products and one of their product is AWS Cloud. Under this product they sell IT infrastructure (storage, memory, network, VM, webhosting etc.)
- Amazon SageMaker is Cloud based Machine Learning Platform, and this is one of the product under AWS Cloud.
- Amazon SageMaker can be used to train AI model, host AI model, monitor the model and hosts many other services which any Data Science project need from data gathering to model serving.
- AWS is oldest cloud service provider in the market.
- AWS Sagemaker was launched in Nov'17.
Google
- Google has hundreds of products like gmail, youtube, google drive etc. One of their product is called Google Cloud. Under this product they sell IT infrastrcture like Amazon sells under AWS.
- VertexAI is Cloud based Machine Learning platform of Google. VertexAI is part of Google Cloud.
- VertexAI can be used to train AI Model,host AI model, monitor the model etc.
- VertexAI was launched in Jun'21
Microsoft
- Like Amazon’s cloud platform which is called AWS Cloud, Microsoft’s cloud plateform is called Azure.
- Microsoft’s AI product is called Azure Machine Learning.
- Today (Jul'23) Azure Machine Learning has has most of the capabilites than any other player’s AI product.
- Azure Machine Learning was launched Feb'14

What is GenAI?
#

There are many kinds of AI models like classifier models, regressor models, clustering models, reinforcement models, etc. An AI model which has the ability to generate text, images, video, and music is called GenAI. They all take inspiration from the human brain, therefore they all have neural network (NN) architecture. There are dozens (if not hundreds) types of NN architecture that can be used to create different kinds of AI models. The type of NN architecture depends upon the data which is used for developing the model and the problem which we want to solve using AI model. Researchers in universities or big corporations like Google, Facebook, Amazon, and Microsoft keep developing new architecture, and using these architectures they develop the foundational models. Once foundational models are developed, they release a research paper. In this, they inform the world what architecture they used, what data they used, what parameters (weights & biases) the model has learned, what are the results of their product and compare that with other existing models. They can develop these foundational models with one set of hyperparameters, and they can release these foundational models of different sizes (it depends upon the number of parameters used). AI product builders pick up these foundational models and fine-tune these based on the exact business problem in their hands. Which foundational model do they choose, it also depends upon the size of the model, the kind of data it has used to create those foundational models, and what was the performance of the model on a similar task which the product developer want to solve.

Types of Machine Learning

hari@dasarpai.com (Dr. Hari Thapliyaal) — Thu, 27 Apr 2023 00:00:00 +0000

Types of Machine Learning
#

Introduction
#

Machine learning is a field of artificial intelligence that focuses on developing algorithms that can learn from data and make predictions or decisions. There are several types of machine learning techniques, each with its strengths and weaknesses. In this post, we will explore some of the most commonly used machine learning techniques, including supervised learning, unsupervised learning, reinforcement learning, and more. This post is not about deep diving into these topics but to give you a oneliner understanding and the difference between these different techniques.

Cost Functions and Optimizers in Machine Learning

hari@dasarpai.com (Dr. Hari Thapliyaal) — Wed, 01 Feb 2023 00:00:00 +0000

Cost-Functions-and-Optimizers-in-Machine-Learning
#

What is machine learning?
#

Machine learning is a subfield of artificial intelligence that focuses on the development of algorithms and statistical models that enable computers to improve their performance on a specific task through experience.

In machine learning, the goal is to develop models that can automatically learn patterns and relationships in data, and use that knowledge to make predictions or take actions. The models are trained on a large dataset, and the learning process involves optimizing the parameters of the model to minimize the prediction error. For this purpose every algorithms uses some cost function or loss function.

Introduction to Neural Network

hari@dasarpai.com (Dr. Hari Thapliyaal) — Tue, 17 Jan 2023 00:00:00 +0000

Introduction to Neural Network
#

Introduction to a Perceptron
#

A perceptron is a type of artificial neural network that can be used for binary classification. It is a simple model that consists of a single layer of artificial neurons and is used to classify input data into one of two categories. The perceptron algorithm learns the weights of the artificial neurons by adjusting them based on the input data and the desired output. The perceptron is considered a basic building block for more complex neural networks.

Linear Regression Interview Questions

hari@dasarpai.com (Dr. Hari Thapliyaal) — Sat, 07 Jan 2023 15:50:00 +0530

Linear Regression Interview Questions and Answers
#

In this question-answer article, I will try that the start of every answer from example rather than theory (some unavoidable variation may be possible). I firmly believe if examples are clear, human mind is smart enough in generlization and creating theories.

Question 1: What is linear regression? What is the difference between simple linear regression and multiple linear regression?
#

Linear regression is a statistical method used to model the linear relationship between a dependent variable and one or more independent variables. It is used to predict the value of the dependent variable based on the values of the independent variables.

Domain Knowledge in Machine Learning

hari@dasarpai.com (Dr. Hari Thapliyaal) — Sat, 15 Oct 2022 15:50:00 +0530

Domain Knowledge in Machine Learning
#

Let’s say the domain is a restaurant kitchen. A dataset with 3 variables. Two predictors and one predicted. Predictor variables are flour in kilograms and water in liters. A predicted variable is the number of roti/ bread. You know the model will be something like this.

Confusion Matrix Bayesian Theorem

hari@dasarpai.com (Dr. Hari Thapliyaal) — Mon, 22 Aug 2022 15:50:00 +0530

If you are like me then you must have struggled enough to understand the confusion matrix or still struggling to understand the metrics of this confusion matrix. A simple 2×2 actual vs predicted values matrix can be looked at and analyzed from different angles and hence we can have all different metrics from this. The beauty of all these metrics is different metrics are important for evaluating different kinds of models.

Folder Structure for ML Project

hari@dasarpai.com (Dr. Hari Thapliyaal) — Mon, 20 Sep 2021 15:50:00 +0530

Directory Structure for ML Project is critical for any serious data science project. What we learn from a technology college and institution is useful in a real-world business, but the problem is that environment is completely different, and it takes proper guidance to adapt to a real business environment. Many times traveling on this path is full of frustration without any proper guidance. Sometimes guidance is there, but it is like blind is leading blind. One has already navigated a lot with struggle, so she/he is trying to guide a new person. Data-science education is not any different from this. From universities and institutions, we learn many concepts, algorithms, art, and science of developing models, etc. but when our customer or management tells us a problem with a simplistic dataset then we don’t know how to proceed. Although there are many templates and frameworks to approach, yet there is no simple cookie-cutter solution for all kinds of problems. Depending upon project duration, resources (human + machine + software), client, and his/her need, the technology available at hand we can choose different approaches.

Generalized AI Model for Prediction

hari@dasarpai.com (Dr. Hari Thapliyaal) — Fri, 17 Sep 2021 15:50:00 +0530

Can we really Develop AI solutions that can predict human behavior? If you are not a technical person then don’t get overwhelmed by the next paragraph, you can read further, and it will make sense to you.

We know the basic equation, y = mx + c. This comes from algebra and trigonometry. Here, y is the predicted value, and x is the input. The x can be a simple scalar value or a vector. Similarly, m is the coefficient in this equation, and it can be a simple scalar value or a vector. If m or x is a vector, then it can hold multiple values. The value of m corresponding to x is also called slope in trigonometry. If a plane is 2 dimensional, then you have one m and one x. But if a plane is complex, and it has, let us say, 10 dimensions then it has 9 m and 9 x. 10th dimensions is predicted by these 9 m and 9 x, using the earlier formula. How that multiplication happens is easy for those who know vector and matrix multiplication, but for others, it is really complicated. So, you can leave it for the time being.

AI ML Resources from My Diary

hari@dasarpai.com (Dr. Hari Thapliyaal) — Wed, 21 Jul 2021 15:50:00 +0530

AI ML Resources from My Diary
#

This is my personal diary which contains resources, which I know, learned or people have told me to experiment with. I started writing this diary in Mar’ 2019. This diary is related to my work/learning in data science, AI, ML, NLP, DL, GNN, GAN, Statistics, etc. A few links related to Software Development, People Management, Project Management are also kept in this diary. The resources here are python/R library link, blog articles, YouTube video links, article links, AI products links, architecture, images etc. The Table of Content and Content of this diary is not any specific order. Whatever I was/am getting I keep adding into this, mostly towards the end but not always.

Data Scientists and AI, ML Researchers

hari@dasarpai.com (Dr. Hari Thapliyaal) — Sat, 17 Jul 2021 00:00:00 +0000

Data Scientists and AI, ML Researchers
#

For the initial list, I have taken content from github. I have expanded this and will keep updating this in the future. If you know any, please feel free to put the name in the comment box.

Machine Learning Tasks and Model Evaluation

hari@dasarpai.com (Dr. Hari Thapliyaal) — Wed, 14 Jul 2021 00:00:00 +0000

Machine Learning Tasks and Model Evaluation
#

Introduction
#

Machine learning is a subject where we study how to create & evaluate machine learning models. To create these models, we need different types of data. We build models which can help us do various kinds of tasks. There are hundreds of model building techniques and researchers keep adding new techniques, and architectures as when need arises. But, the question is how do you evaluate these models which are output of the model trainings? To evaluate the performance of a model on structured data, or classification/regression/clustering models, we require one kind of metrics. But this becomes complicated when we are dealing with voice, text and audio data. How do you evaluate ten models which are responsible for translation, or locating an object in the image, transcribing voice into text, captioning an image? To solve this problem, standard databases are created and everyone needs to demonstrate the performance of their model, architecture, or approach against that dataset. But, even if you have a baseline dataset, how will you evaluate various NLP or deep learning tasks? For that GLUE, SuperGLUE benchmarks are created.

Machine Learning Framework, Library, Tools

hari@dasarpai.com (Dr. Hari Thapliyaal) — Tue, 13 Jul 2021 00:00:00 +0000

Machine Learning Framework, Library, Tools
#

Introduction
#

As of 2022, Data Science, AI, and Machine Learning are very fast-evolving domains. In the last 5 years, it has picked up momentum. Every day a new library, framework, and tools pops out before our eyes. Many of these are very interesting and some are just duplicates with little value addition. I am listing below some tools that I came across during my work. The purpose of this listing is not to describe every tool but to provide a link and just put them in sorted order. There are so many jargons, and we use to forget these. Sometimes, it becomes difficult to recall them. It was useful for me and I hope some of you will find this useful.

AI, ML, DL Blogs Sites

hari@dasarpai.com (Dr. Hari Thapliyaal) — Mon, 12 Jul 2021 00:00:00 +0000

AI, ML, DL Blogs Sites
#

370+ blogs/website, who publishes articles, research, opinions, observations related to AI, ML, DL, Signal Processing, Computer Vision, Text Processing, Audio Processing, Image/video/Audio/Text Generation on regular basis.

1 101.datasciencemunity
2 2020.emnlp.org
3 24x7coach
4 aaai.org
5 academic.microsoft
6 aclweb.org
7 ai.facebook
8 ai.sri
9 ai4bharat.org
10 aiai.ed.ac.uk
11 ai-junkie
12 aiweekly.co
13 algorithmica-technologies
14 allenai.org
15 allthingsds.wordpress
16 amitness
17 analytics.google
18 analyticsindiamag
19 analyticstraining
20 analyticsvidhya
21 apache.org
22 archive.ics.uci.edu
23 arxiv.org
24 arxiv-sanity
25 atom.io
26 becomingadatascientist
27 blockchain
28 blog.cloudera
29 blog.datadive.net
30 blog.data-miners
31 blog.dominodatalab
32 blog.gramener
33 blog.kaggle
34 blog.mendeley
35 blog.pythonlibrary.org
36 blog.revolutionanalytics
37 blog.smola.org
38 blogs.sun
39 bokeh.pydata.org
40 brilliant.org
41 bsc.hcverma.in
42 byjus
43 calculatedriskblog
44 campus.datacamp
45 catalog.data.gov
46 censusindia.gov.in
47 chioka.in
48 ci.pytorch.org
49 classroom.udacity
50 cloud.google
51 codecogs
52 community.powerbi
53 community.tableau
54 corenlp.run
55 coursera.org
56 cran.r-project.org
57 cs.brown.edu
58 cs.rochester.edu
59 cs.utexas.edu
60 cs.washington.edu
61 csail.mit.edu
62 cse.unr.edu
63 dair.ai
64 danielforsyth.me
65 dartpad.dartlang.org
66 data.gov.in
67 data.gov.uk
68 data.mendeley
69 data.world
70 databricks
71 datacamp
72 dataconomy
73 data-flair.training
74 datahack.analyticsvidhya
75 dataists
76 datajournalismhandbook.org
77 datalabsagency
78 data-magnum
79 data-mania
80 datamation
81 datanews.tumblr
82 dataplatform.cloud.ibm
83 dataquest.io
84 dataschool.io
85 datascience
86 datasciencecentral
87 datasciencedegreeprograms.net
88 datasciencelab.wordpress
89 datasciencelondon.org
90 datasciencemadesimple
91 datascienceplus
92 datasciencereport
93 datasciencevademecum.wordpress
94 datascienceweekly.org
95 datascientistjourney.wordpress
96 datascientists.net
97 datascopeanalytics
98 datasets.quantumstat
99 datasetsearch.research.google
100 datastori.es
101 datastudio.google
102 datatofish
103 dataversity.net
104 datawrangling
105 deeplearning.ai
106 deeplearning.cs.toronto.edu
107 deeplearning.net
108 deeplearning.stanford.edu
109 deeplearningbook.org
110 degruyter
111 desmos
112 developer.ibm
113 developers.google
114 devops
115 dezyre
116 digitalvidya
117 distill.pub
118 dl.acm.org
119 docs.bokeh.org
120 docs.cltk.org
121 docs.docker
122 docs.fast.ai
123 docs.flutter.io
124 docs.microsoft
125 docs.opencv.org
126 docs.oracle
127 docs.python-guide.org
128 docs.scipy.org
129 docs.sympy.org
130 doi.org
131 drivendata.org
132 eecs.umich.edu
133 emilio.ferrara.name
134 explosion.ai
135 fastml
136 flowingdata
137 freecodecamp.org
138 gapminder.org
139 gartner
140 geeksforgeeks.org
141 gluebenchmark
142 gramener
143 gregreda
144 guide2research
145 guru99
146 hackernoon
147 hackr.io
148 harvarddatascience
149 hbr.org
150 heroku
151 herokuapp
152 hips.seas.harvard.edu
153 howstat
154 huggingface.co
155 i2ocr
156 icsalabs
157 ieee-dataport.org
158 ieeexplore.ieee.org
159 iisc.ac.in
160 iitm.ac.in
161 incompleteideas.net
162 inltk.readthedocs.io
163 inventwithpython
164 investopedia
165 iot-analytics
166 isca-students.org
167 isi.edu
168 itbusinessedge
169 jeffdonahue
170 jmlr.org
171 johnmyleswhite
172 journaldev
173 journals.elsevier
174 kaushik.net
175 kdnuggets
176 kennybastani
177 khanacademy.org
178 kore.ai
179 kubicle
180 latex4technics
181 learninglover
182 learnopencv
183 learnpythonthehardway.org
184 lexalytics
185 lexilogos
186 link.springer
187 listendata
188 louisdorard
189 lukasbiewald
190 machinelearningmastery
191 mapr
192 mastersindatascience.org
193 mathgoodies
194 mathopenref
195 mathopolis
196 mathplanet
197 mathsisfun
198 matplotlib.org
199 mdmgeek
200 medium
201 metabrown
202 micfarris
203 mickaellegal
204 mindsmapped
205 mitpressjournals.org
206 ml-compiled.readthedocs.io
207 mpi-inf.mpg.de
208 mybinder.org
209 mybinder.readthedocs.io
210 naftaliharris
211 namami.gov.in
212 nbviewer.jupyter.org
213 newdatascientist.blogspot
214 news.startup.ml
215 nips.cc
216 nlp.stanford.edu
217 nltk.org
218 nltk.org
219 nplt.in
220 nrl.navy.mil
221 numpy.org
222 oigetit
223 online.tableau
224 online.umich.edu
225 onlinecourses.nptel.ac.in
226 onlinemathlearning
227 open.wolframcloud
228 openaccess.thecvf
229 openai
230 opencv.org
231 opencv-python-tutroals.readthedocs.io
232 opengroup.org
233 opensciencedatacloud.org
234 oreilly
235 pandas.pydata.org
236 pantechsolutions.net
237 pantechsolutions.net
238 paperswithcode
239 paralleldots
240 pathmind
241 pbpython
242 periscopic
243 pewresearch.org
244 portal.azure
245 portal.office
246 powerbi.microsoft
247 practicalquant.blogspot
248 projects.apache.org
249 p-value.info
250 pyimagesearch
251 pymbook.readthedocs.io
252 python.org
253 python-graph-gallery
254 pythonlinks.info
255 pythonprogramming.net
256 quickdraw.withgoogle
257 rapidapi
258 rasa
259 r-bloggers
260 realpython
261 reddit
262 regex101
263 regexone
264 researchgate.net
265 scholar.google.co.in
266 scholar.google
267 sciencedirect
268 sci-hub.tw
269 scikit-image.org
270 scikit-learn.org
271 scl.samsaadhanii.in
272 scribd
273 seaborn.pydata.org
274 searchtechnologies
275 sebastianraschka
276 semanticscholar.org
277 setosa.io
278 sid.iisc.ac.in
279 sigdat.org
280 sigir.org
281 signll.org
282 sjri.res.in
283 smartcities.data.gov.in
284 sotabench
285 sourceforge.net
286 spenczar
287 springboard
288 springer
289 sqlbi
290 stat.ucla.edu
291 statisticshowto.datasciencecentral
292 statlect
293 stats.stackexchange
294 stattrek
295 studio.azureml.net
296 sundog-education
297 superdatascience
298 susiddha.org
299 tableau
300 tarrysingh
301 tcdata360.worldbank.org
302 techcrunch
303 techgig
304 technologyreview
305 techrepublic
306 tensorflow.org
307 theaisummer
308 theblog.okcupid
309 theinsaneapp
310 the-scientist
311 thisismetis
312 tn.data.gov.in
313 towardsdatascience
314 translate.google
315 trends.google
316 tutorialandexample
317 tutorialspoint
318 varsitytutors
319 vistawide
320 visualqa.org
321 visualstudio
322 w3schools
323 wandb
324 whatsthebigdata
325 wiki.python.org
326 wikicfp
327 wolframalpha
328 wordnet.princeton.edu
329 www-aig.jpl.nasa.gov
330 yerevann
331 yet-another-data-blog.blogspot.tr
332 ztable.net
333 excel.tips.net
334 excelcampus
335 exceleratorbi.au
336 intellipaat
337 knowledgehut
338 leaps.analyttica
339 learn.byjus
340 learndigital.withgoogle
341 learning.edureka.co
342 udacity
343 udemy
344 anaconda
345 codegrepper
346 codepad.org
347 codingbat
348 colab.research.google
349 docs.python.org
350 interactivepython.org
351 julialang.org
352 jupyter.org
353 kaggle
354 learnpython.org
355 paperspace
356 pytorch.org
357 adeshpande3.github.io
358 harvard-iacs.github.io
359 iamtrask.github.io
360 kiwidamien.github.io
361 ronxin.github.io
362 spandan-madan.github.io
363 stanfordnlp.github.io
364 trigonaminima.github.io
365 Interview Query
366 Hackerrank
367 CrackingTheCodingInterview
368 Google Research
369 Google AI Blog
370 ML Code Completeness Checklist,
371 ML Reproducibility Checklist
372 Huggingface Transformer

AI, ML, Deep Learning, NLP Conferences & Journals

hari@dasarpai.com (Dr. Hari Thapliyaal) — Thu, 08 Jul 2021 00:00:00 +0000

AI, ML, Deep Learning, NLP Conferences & Journals
#

Content on this page keeps changing. These links are taken from my chrome browser favorites.

A list of 30+ AI conferences and journals related to AI, NLP, Deep Learning. Almost all major happening in the world of AI/ML is presented in these conferences and published in these magazines.

How Naive Bayes Classifier Works

hari@dasarpai.com (Dr. Hari Thapliyaal) — Wed, 31 Mar 2021 15:50:00 +0530

How Naive Bayes Classifier Works?
#

Naive Bayes classifier example
#

In this presentation, I am not going into the depth of the Naive Bayes algorithm. I am assuming you have heard this term many times but are not able to visualize it mentally or struggling to comprehend this. If that is the case, then you are on the right page.

EDA & Feature Engineering 101

hari@dasarpai.com (Dr. Hari Thapliyaal) — Mon, 24 Aug 2020 15:50:00 +0530

What is EDA?
#

EDA means Exploratory Data Analysis. The purpose of data analysis is to explore. Exploration means try to understand what kind of data I have in my hand. Using EDA we try to get the answer to the following questions.

What kind of data is this? (file format, volume of data, number of columns, metadata data of image/video/audio or some feedback in English or other languages, or tabular data, etc)
How complex is this data? (How many files are there? primary key? how these files are connected to each other? is nested data in some files? is some field having nested data, etc.)
Is this data sufficient for meeting our ultimate goal, i.e. Model building?
Is there any missing data? Data needed but not given by the business or not available at all or costly to get that data etc.
Are there any missing values? In the given dataset do we have complete information or some values are missing for some records or some columns?
What are different independent and dependent fields?
Is there any relationship between different independent variables of the dataset? If yes then how strong is that relationship?
Are observations independent or tightly coupled like we see in time-series data?

In the data scient project lifecycle, EDA is not a sequential, one-time, isolated process. Till the time data is not ready for modeling we keep doing EDA and cleaning the data. So, EDA is followed by a list of decisions taken to clean the dataset, and finally, data cleaning steps are implemented. If the dataset is not in the good shape after the first iteration of EDA we continue EDA in the next cycle. In this article, I am not referring to EDA as just visualizing and understanding the dataset but all the steps required till the dataset is not ready for modeling.

DS, AI, ML Online Course, Tutorial, Videos

hari@dasarpai.com (Dr. Hari Thapliyaal) — Thu, 02 Jul 2020 00:00:00 +0000

DS, AI, ML Online Course, Tutorial, Videos
#

Courses
#

Machine Learning – Stanford by Andrew Ng in Coursera (2010-2014)
Machine Learning – Caltech by Yaser Abu-Mostafa (2012-2014)
Machine Learning – Carnegie Mellon by Tom Mitchell (Spring 2011)
Neural Networks for Machine Learning by Geoffrey Hinton in Coursera (2012)
Neural networks class by Hugo Larochelle from Université de Sherbrooke (2013)
Deep Learning Course by CILVR lab @ NYU (2014)
A.I – Berkeley by Dan Klein and Pieter Abbeel (2013)
A.I – MIT by Patrick Henry Winston (2010)
Vision and learning – computers and brains by Shimon Ullman, Tomaso Poggio, Ethan Meyers @ MIT (2013)
Convolutional Neural Networks for Visual Recognition – Stanford by Fei-Fei Li, Andrej Karpathy (2017)
Deep Learning for Natural Language Processing – Stanford
Neural Networks – usherbrooke
Machine Learning – Oxford (2014-2015)
Deep Learning – Nvidia (2015)
Graduate Summer School: Deep Learning, Feature Learning by Geoffrey Hinton, Yoshua Bengio, Yann LeCun, Andrew Ng, Nando de Freitas and several others @ IPAM, UCLA (2012)
Deep Learning – Udacity/Google by Vincent Vanhoucke and Arpan Chakraborty (2016)
Deep Learning – UWaterloo by Prof. Ali Ghodsi at University of Waterloo (2015)
Statistical Machine Learning – CMU by Prof. Larry Wasserman
Deep Learning Course by Yann LeCun (2016)
Designing, Visualizing and Understanding Deep Neural Networks-UC Berkeley
UVA Deep Learning Course MSc in Artificial Intelligence for the University of Amsterdam.
MIT 6.S094: Deep Learning for Self-Driving Cars
MIT 6.S191: Introduction to Deep Learning
Berkeley CS 294: Deep Reinforcement Learning
Keras in Motion video course
Practical Deep Learning For Coders by Jeremy Howard – Fast.ai
Introduction to Deep Learning by Prof. Bhiksha Raj (2017)
AI for Everyone by Andrew Ng (2019)
MIT Intro to Deep Learning 7 day bootcamp – A seven day bootcamp designed in MIT to introduce deep learning methods and applications (2019)
Deep Blueberry: Deep Learning – A free five-weekend plan to self-learners to learn the basics of deep-learning architectures like CNNs, LSTMs, RNNs, VAEs, GANs, DQN, A3C and more (2019)
Spinning Up in Deep Reinforcement Learning – A free deep reinforcement learning course by OpenAI (2019)
Deep Learning Specialization – Coursera – Breaking into AI with the best course from Andrew NG.
Deep Learning – UC Berkeley STAT-157 by Alex Smola and Mu Li (2019)
Machine Learning for Mere Mortals video course by Nick Chase
Machine Learning Crash Course with TensorFlow APIs -Google AI
Deep Learning from the Foundations Jeremy Howard – Fast.ai
Deep Reinforcement Learning (nanodegree) – Udacity a 3-6 month Udacity nanodegree, spanning multiple courses (2018)
Grokking Deep Learning in Motion by Beau Carnes (2018)
Face Detection with Computer Vision and Deep Learning by Hakan Cebeci
Presentation skills: Designing Presentation Slides - Coursera
Mathematics for Machine Learning: Multivariate Calculus - Coursera
Machine Learning – Home Coursera
Mathematics for Machine Learning: Multivariate Calculus - Coursera
Data Science Certificates - Coursera
Edureka
Edureka-Cloudera Manager
Udemy Courses
Courses – Online Reiki Course
DataCamp Courses
Byju
udacity
What is Spark – A Comparison Between Spark vs. Hadoop
Microsoft Azure Machine Learning Studio (classic)
Welcome to The Apache Software Foundation!
Making India Employable - Vivid Vision 10 10 10
GpI8H5 – Online Python3 Interpreter & Debugging Tool – Ideone.com
Google I/O 2019 – All Sessions – YouTube
TensorFlow at Google I/O 2019 – YouTube
Quantum Mechanics - BSc Lectures by Prof. H C Verma and Team
Open Pathshala - Your Best Source to Learn Sanskrit
Class Central #1 Search Engine for Free Online Courses & MOOCs
Free Online Course: Mathematics for Machine Learning: Multivariate Calculus from Coursera - Class Central
e Learning for Basic Science and Maths
Online Classes by Skillshare - Start for Free Today
Learn online marketing with free courses – Google Digital Garage
Moz Blog – SEO and Inbound Marketing Blog – Moz
NPTEL Online Courses Mobile
Learn Python, Data Viz, Pandas & More - Tutorials - Kaggle
Data Science Training

Tutorials
#

Videos and Lectures
#

How To Create A Mind By Ray Kurzweil
Deep Learning, Self-Taught Learning and Unsupervised Feature Learning By Andrew Ng
Recent Developments in Deep Learning By Geoff Hinton
The Unreasonable Effectiveness of Deep Learning by Yann LeCun
Deep Learning of Representations by Yoshua bengio
Principles of Hierarchical Temporal Memory by Jeff Hawkins
Machine Learning Discussion Group – Deep Learning w/ Stanford AI Lab by Adam Coates
Making Sense of the World with Deep Learning By Adam Coates
Demystifying Unsupervised Feature Learning By Adam Coates
Visual Perception with Deep Learning By Yann LeCun
The Next Generation of Neural Networks By Geoffrey Hinton at GoogleTechTalks
The wonderful and terrifying implications of computers that can learn By Jeremy Howard at TEDxBrussels
Unsupervised Deep Learning – Stanford by Andrew Ng in Stanford (2011)
Natural Language Processing By Chris Manning in Stanford
A beginners Guide to Deep Neural Networks By Natalie Hammel and Lorraine Yurshansky
Deep Learning: Intelligence from Big Data by Steve Jurvetson (and panel) at VLAB in Stanford.
Introduction to Artificial Neural Networks and Deep Learning by Leo Isikdogan at Motorola Mobility HQ
NIPS 2016 lecture and workshop videos – NIPS 2016
Deep Learning Crash Course: a series of mini-lectures by Leo Isikdogan on YouTube (2018)
Deep Learning Crash Course By Oliver Zeigermann
Deep Learning with R in Motion: a live video course that teaches how to apply deep learning to text and images using the powerful Keras library and its R language interface.
8 Essential Tips for People starting a Career in Data Science.
Cheatsheet: How to become a data scientist.
The Art of Learning Data Science.
The Periodic Table of Data Science.
Aspiring Data Scientists! Start to learn Statistics with these 6 books!
8 Skills You Need to Be a Data Scientist
Top 10 Essential Books for the Data Enthusiast
Aspiring data scientist? Master these fundamentals.
How to Become a Data Scientist – On your own.

GRETL – Great Statistical software for Beginners
#

Simple Linear Regression https://lnkd.in/ecfsV9c
Coding Dummy Variables https://lnkd.in/ef7Yd7f
Forecasting New Observations https://lnkd.in/eNKbxbU
Forecasting a Large Number of Observations https://lnkd.in/eHmibGs
Logistic Regression https://lnkd.in/eRfhQ87
Forecasting and Confusion Matrix https://lnkd.in/eaqrFJr
Modeling and Forecasting Time Series Data https://lnkd.in/e6fqKpF
Comparing Time Series Trend Models https://lnkd.in/eKjEUAE
Khan Academy is the best online free resource to learn Math for Data Science. ( https://www.khanacademy.org/math/).
Krista King has also done a great job in creating an exceptionally good introductory course. She is too good at designing the course. ( https://www.udemy.com/user/kristaking/.
3Blue1Brown ( https://www.youtube.com/channel/UCYO_jab_esuFRV4b17AJtAw/playlists).
Every Intro to Data Science Course on the Internet, Ranked. (https://lnkd.in/fQDMiNX )
What would be useful for aspiring data scientists to know? (https://lnkd.in/fmcFyN7)

Reinforcement Learning Git Repositories

hari@dasarpai.com (Dr. Hari Thapliyaal) — Wed, 01 Jul 2020 00:00:00 +0000

Reinforcement Learning Git Repositories
#

Sno.	URL	Description
1	https://github.com/openai/baselines	OpenAI Baselines: high-quality implementations of reinforcement learning algorithms
2	https://github.com/hill-a/stable-baselines	A fork of OpenAI Baselines, implementations of reinforcement learning algorithms
3	https://github.com/openai/spinningup	An educational resource to help anyone learn deep reinforcement learning.
4	https://github.com/google/dopamine	Dopamine is a research framework for fast prototyping of reinforcement learning algorithms.
5	https://github.com/tensorflow/agents	TF-Agents is a library for Reinforcement Learning in TensorFlow
6	https://github.com/deepmind/trfl	TensorFlow Reinforcement Learning
7	https://github.com/facebookresearch/Horizon	A platform for Applied Reinforcement Learning (Applied RL)
8	https://github.com/facebookresearch/ELF	An End-To-End, Lightweight and Flexible Platform for Game Research
9	https://github.com/NervanaSystems/coach	Reinforcement Learning Coach by Intel AI Lab enables easy experimentation with state of the art Reinforcement Learning algorithms
10	https://github.com/ray-project/ray/tree/master/python/ray/rllib	A fast and simple framework for building and running distributed applications.
11	https://github.com/keras-rl/keras-rl	Deep Reinforcement Learning for Keras.
12	https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail	PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).
13	https://github.com/Kaixhin/Rainbow	Rainbow: Combining Improvements in Deep Reinforcement Learning
14	https://github.com/MillionIntegrals/vel	Velocity in deep-learning research
15	https://github.com/tensorforce/tensorforce	Tensorforce: A TensorFlow library for applied reinforcement learning
16	https://github.com/kengz/SLM-Lab	Modular Deep Reinforcement Learning framework in PyTorch.
17	https://github.com/rlworkgroup/garage	A framework for reproducible reinforcement learning research
18	https://github.com/catalyst-team/catalyst	Reproducible and fast DL & RL.
19	https://github.com/higgsfield/RL-Adventure	Pytorch Implementation of DQN / DDQN / Prioritized replay/ noisy networks/ distributional values/ Rainbow/ hierarchical RL
20	https://github.com/qfettes/DeepRL-Tutorials	Contains high quality implementations of Deep Reinforcement Learning algorithms written in PyTorch
21	https://github.com/openai/gym	A toolkit for developing and comparing reinforcement learning algorithms.
22	https://github.com/deepmind/lab	A customisable 3D platform for agent-based AI research
23	https://github.com/Microsoft/malmo	Project Malmo is a platform for Artificial Intelligence experimentation and research built on top of Minecraft. We aim to inspire a new generation of research into challenging new problems presented by this unique environment. — For installation instruct
24	https://github.com/openai/retro	Retro Games in Gym
25	https://github.com/deepmind/dm_control	The DeepMind Control Suite and Package
26	https://github.com/openai/neural-mmo	Neural MMO – A Massively Multiagent Game Environment
27	https://github.com/openai/gym	Gym @ OpenAI
28	https://github.com/deepmind/lab	Lab @ DeepMind
29	https://github.com/Microsoft/malmo	Project Malmo @ Microsoft
30	https://github.com/openai/retro	Retro @ OpenAI
31	https://github.com/deepmind/dm_control	Control Suite @ DeepMind
32	https://github.com/openai/neural-mmo	Neural MMO @ OpenAI
33	https://github.com/openai/baselines	Tensorflow Maintained by OpenAI
34	https://github.com/hill-a/stable-baselines	Tensorflow Maintained by Antonin Raffin, Ashley Hill
35	https://github.com/catalyst-team/catalyst	PyTorch Maintained by Sergey Kolesnikov
36	https://github.com/ray-project/ray/tree/master/python/ray/rllib	Tensorflow Maintained by Ray Team
37	https://github.com/tensorflow/agents	Tensorflow Maintained by Google
38	https://github.com/facebookresearch/Horizon	PyTorch Maintained by Facebook
39	https://github.com/NervanaSystems/coach	Tensorflow Maintained by Intel
40	https://github.com/rlworkgroup/garage	Tensorflow Maintained by community
41	https://github.com/kengz/SLM-Lab	PyTorch Maintained by Wah Loon Keng, Laura Graesser
42	https://github.com/google/dopamine	Tensorflow Maintained by Google
43	https://github.com/openai/spinningup	Tensorflow Maintained by OpenAI
44	https://github.com/deepmind/trfl	Tensorflow Maintained by DeepMind
45	https://github.com/deepmind/scalable_agent	Tensorflow Maintained by DeepMind
46	https://github.com/facebookresearch/ELF	PyTorch Maintained by Facebook
47	https://github.com/keras-rl/keras-rl	Tensorflow Maintained by Matthias Plappert
48	https://github.com/ikostrikov/pytorch-a2c-ppo-acktr-gail	PyTorch Maintained by Ilya Kostrikov
49	https://github.com/Kaixhin/Rainbow	PyTorch Maintained by Kai Arulkumaran
50	https://github.com/MillionIntegrals/vel	PyTorch Maintained by Jerry (?)
51	https://github.com/Khrylx/PyTorch-RL	PyTorch
52	https://github.com/tensorforce/tensorforce	Tensorflow
53	https://github.com/higgsfield/RL-Adventure	PyTorch
54	https://github.com/qfettes/DeepRL-Tutorials	PyTorch
55	https://github.com/SurrealAI/surreal	TorchX
56	https://github.com/zuoxingdong/lagom	PyTorch
57	https://github.com/dennybritz/reinforcement-learning	Tensorflow
58	https://github.com/unixpickle/anyrl-py	Tensorflow
59	https://github.com/Scitator/rl-course-experiments	Tensorflow
60	https://github.com/oxwhirl/pymarl	PyTorch

Machine Learning on

Exploring Reinforcement Learning Concepts: A Comprehensive Guide

Exploring Reinforcement Learning Concepts #

Key Concepts of Reinforcement Learning (RL) #

1. Markov Decision Processes (MDPs) #

2. Policies #

3. Value Functions #

4. Exploration vs. Exploitation #

5. Algorithms #

6. Function Approximation #

7. Temporal Difference (TD) Learning #

8. Eligibility Traces #

9. Multi-Agent RL (MARL) #

10. Transfer Learning in RL #

11. Safe and Ethical RL #

12. Hierarchical RL (HRL) #

13. Imitation Learning #

14. Meta-Learning in RL #

15. Exploration Strategies #

16. Challenges in RL #

17. Applications of RL #

18. Tools and Frameworks #

19. Theoretical Foundations #

20. Advanced Topics #

What are differening rewardng systems in RL? #

Exploring Types of Models

Understanding Different Types of Models #

Introduction #

AI ML Project Ideas

AI Project Ideas #

Introduction #

Transfer Learning Key AI Techniques Explained

Transfer Learning Key AI Techniques Explained #

Prompt Engineering, In Context Learning and Zero-shot Learning #

What is in-context Learning? #

Machine Learning Key Concepts

Machine Learning Key Concepts #

Is this article for me? #

Automated Machine Learning

Automated Machine Learning #

Is this article for me? #

Tensorflow GPU Setup on Local Machine

Tensorflow GPU Setup on Local Machine #

Introduction #

MLOps Tools

MLOps Tools #

Why to Finetune LLM?

Finetuning, Fewshot Learning, Why and How? #

Why to finetune a LLM? #

Dimensionality Reduction and Visualization

Dimensionality Reduction and Visualization #

What are the popular methods of dimensionality reduction? #

ML Model Development Framework

ML Model Development Framework & Model Repositories #

Introduction #

ML Model Respository from Pinto0309

ML Model Repository from Pinto0309 #

Introduction #

Distances in Machine Learning

Distances in Machine Learning #

Machine Learning Metrics

Machine Learning Metrics #

Introduction #

Introduction to ML Model Deployment

Introduction to AI Model deployment #

Big Players #

What is GenAI? #

Types of Machine Learning

Types of Machine Learning #

Introduction #

Cost Functions and Optimizers in Machine Learning

Cost-Functions-and-Optimizers-in-Machine-Learning #

What is machine learning? #

Introduction to Neural Network

Introduction to Neural Network #

Introduction to a Perceptron #

Linear Regression Interview Questions

Linear Regression Interview Questions and Answers #

Question 1: What is linear regression? What is the difference between simple linear regression and multiple linear regression? #

Domain Knowledge in Machine Learning

Exploring Reinforcement Learning Concepts
#

Key Concepts of Reinforcement Learning (RL)
#

1. Markov Decision Processes (MDPs)
#

2. Policies
#

3. Value Functions
#

4. Exploration vs. Exploitation
#

5. Algorithms
#

6. Function Approximation
#

7. Temporal Difference (TD) Learning
#

8. Eligibility Traces
#

9. Multi-Agent RL (MARL)
#

10. Transfer Learning in RL
#

11. Safe and Ethical RL
#

12. Hierarchical RL (HRL)
#

13. Imitation Learning
#

14. Meta-Learning in RL
#

15. Exploration Strategies
#

16. Challenges in RL
#

17. Applications of RL
#

18. Tools and Frameworks
#

19. Theoretical Foundations
#

20. Advanced Topics
#

What are differening rewardng systems in RL?
#

Understanding Different Types of Models
#

Introduction
#

AI Project Ideas
#

Introduction
#

Transfer Learning Key AI Techniques Explained
#

Prompt Engineering, In Context Learning and Zero-shot Learning
#

What is in-context Learning?
#

Machine Learning Key Concepts
#

Is this article for me?
#

Automated Machine Learning
#

Is this article for me?
#

Tensorflow GPU Setup on Local Machine
#

Introduction
#

MLOps Tools
#

Finetuning, Fewshot Learning, Why and How?
#

Why to finetune a LLM?
#

Dimensionality Reduction and Visualization
#

What are the popular methods of dimensionality reduction?
#

ML Model Development Framework & Model Repositories
#

Introduction
#

ML Model Repository from Pinto0309
#

Introduction
#

Distances in Machine Learning
#

Machine Learning Metrics
#

Introduction
#

Introduction to AI Model deployment
#

Big Players
#

What is GenAI?
#

Types of Machine Learning
#

Introduction
#

Cost-Functions-and-Optimizers-in-Machine-Learning
#

What is machine learning?
#

Introduction to Neural Network
#

Introduction to a Perceptron
#

Linear Regression Interview Questions and Answers
#

Question 1: What is linear regression? What is the difference between simple linear regression and multiple linear regression?
#

Domain Knowledge in Machine Learning
#

AI ML Resources from My Diary
#

Data Scientists and AI, ML Researchers
#

Machine Learning Tasks and Model Evaluation
#

Introduction
#

Machine Learning Framework, Library, Tools
#

Introduction
#

AI, ML, DL Blogs Sites
#

AI, ML, Deep Learning, NLP Conferences & Journals
#

How Naive Bayes Classifier Works?
#

Naive Bayes classifier example
#

What is EDA?
#

DS, AI, ML Online Course, Tutorial, Videos
#

Courses
#

Tutorials
#

Videos and Lectures
#

GRETL – Great Statistical software for Beginners
#

Reinforcement Learning Git Repositories
#