Chevron Left
Back to Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

Learner Reviews & Feedback for Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization by DeepLearning.AI

4.9
stars
62,864 ratings

About the Course

In the second course of the Deep Learning Specialization, you will open the deep learning black box to understand the processes that drive performance and generate good results systematically. By the end, you will learn the best practices to train and develop test sets and analyze bias/variance for building deep learning applications; be able to use standard neural network techniques such as initialization, L2 and dropout regularization, hyperparameter tuning, batch normalization, and gradient checking; implement and apply a variety of optimization algorithms, such as mini-batch gradient descent, Momentum, RMSprop and Adam, and check for their convergence; and implement a neural network in TensorFlow. The Deep Learning Specialization is our foundational program that will help you understand the capabilities, challenges, and consequences of deep learning and prepare you to participate in the development of leading-edge AI technology. It provides a pathway for you to gain the knowledge and skills to apply machine learning to your work, level up your technical career, and take the definitive step in the world of AI....

Top reviews

AB

Aug 26, 2021

Amazing course which focus on the theoretical part of parameters tuning, but it needs more explanation of Tensorflow, as I felt a little lost in the last project. Except that, it is an amazing course.

AS

Apr 18, 2020

Very good course to give you deep insight about how to enhance your algorithm and neural network and improve its accuracy. Also teaches you Tensorflow. Highly recommend especially after the 1st course

Filter by:

6026 - 6050 of 7,219 Reviews for Improving Deep Neural Networks: Hyperparameter Tuning, Regularization and Optimization

By Oleg P

Nov 25, 2018

This was a good course with several drawbacks. Although we learned theoretical aspects of batch normalization, I needed to go through a dozen of resources to learn how to actually implement it (especially backpropagation part). An assignment for that would be helpful. The last week of this course (Tensorflow) should have become its own course with multiply programming assignments. What I strongly dislike is the design of programming assignments, which are very long but the user implementation part very short - it's easy to loose an overview. Personally, I finish them as quick as possible to get the grade and then implement the functionality by myself. I'd also support the idea of optional assignments, where students can solve some problems from scratch.

By Volodymyr M

Apr 18, 2020

I am more, or less satisfied with this part of the course. It gives valuable information in many practical aspects. What is not good - graded assignments notebooks contain errors. For me, having about 30 years of experience in various technical fields, it was like an interesting game to find source of error and make graded assignment running, while some other people may just get stuck.

Also, I wish we run graded assignments on computers with a little bit more power. It is very unproductive to wait tens of seconds while Python kernel evaluates the cell.

BTW, for me with week 3 graded assignment first 10 epochs took about 10 minutes. Simple math gives 2.5 hours for 1500 epochs. Yes, we are paying not that much for this course, but we are paying for it.

By Joseph C

Mar 30, 2018

This was another fantastic course in the deep learning specialization. We are introduced to techniques designed to speed up and improve neural network models, such as minibatch gradient descent, regularization, and normalization techniques. Hands-on programming exercises show how to implement some of these techniques. Suggestions for improvement: (1) Third week is too dense with ideas. It should provide programming exercises to impart practical understanding of these ideas. A 4th week should be devoted to an in-depth introduction to TensorFlow. (2) Improve the Forum by including regular participation by the teaching staff. Mentors are active on the Forum, but there is virtually NO participation by the teaching staff!

By Yuri G

May 12, 2018

It's a very useful course but it comes across as the creators have put it out and completely abandoned it: the contact email given at the beginning is a black hole (could have used no_reply), volunteer mentors are supporting the forums but well-known multiple bugs and mistakes are not being fixed. Programming assignments are designed to make an impression that students can reach farther than they actually can on their own by often turning them into a copy-and-paste of small pieces into the code already mostly written for them. One would expect that if one even needs to use this "knowledge" from programming assignments for work later it would be free and readily available. Wrong assumption.

By Mehran M

Jun 7, 2018

I really really liked both this course and the previous one. In fact, I gave 5 stars to the first course. However, compared to the first course, this one earns 4 stars in my book. For one, I wish there was more material on TensorFlow. Also it would've been nice to do a detailed assignment involving actual hyperparameter tuning. Even though the title of the course promises techniques about hyperparameter tuning, I'm not confident that I will be able to do a systematically good job, even though I just finished this course and have carefully taken notes of all the presented materials. Nevertheless, this is an awesome course and I wholeheartedly recommend it.

By Brian R

Dec 20, 2018

This course is a gem in that its chock-a-block full of real-world/in-practice information that so many courses don't cover. One note is that some of the programming assignment notebooks in this course (especially the TF one) were several notches below the finely tuned level of the prior course's. There were typos in the explanation sections, syntax errors in blocks so that you couldn't do partial submissions, and other things that didn't create a full-stop but definitely upped the frustration factor. I made it through and have learned a MASSIVE amount, but there is definitely some polishing that would have made this a 5 star that wasn't there yet.

By Enrique C M

Oct 18, 2017

Very good course about more critical concepts when building deep neural networks. Although the material seems like quite condensed and forced in a very short span of time (3 weeks) while the easier and more basic concepts in course 1 where explained at the right pace during 4 weeks.

These are core concepts and techniques for practical day-by-day deep learning engineering and programming and I would have wanted them to not to be taught in such a rush.

Even though, highly recommended course... I am already building real networks to solve real problems in some projects I am involved in and that is simply awesome after 2 courses of this specialization :)))

By Ravish C

Oct 2, 2017

The content was very well taught and I learnt a lot of new concepts I was not familiar with before. I liked that the projects had us implement the basic concepts like Adam optimizer and Dropout on a neural network from scratch, instead of with tensorflow because it helped me understand how they work essentially.

That said, it felt that the course was less polished as the first one. There were some errors in the assignments that made them difficult to pass, the assignments themselves were also very easy and did not take as long to complete (at most I spent 30 minutes on any assignment, instead of the written 2 hours).

By Categorical S

Dec 23, 2018

Great course overall. Thanks to the instructor and TA's and mentors on the forums for making this possible.

For students my notes would be - a) a bit too much hand holding on assignments and b) the subject matter is less interesting conceptually (unless you are into numerical optimization). For instance- I skipped ahead to RNNs to keep it fresh.

For staff my feedback would be : consider re-shuffling the course material into the other courses (CNN , RNN) rather than have this all in a block. The subject matter is a bit dry, and for instance, tensor flow can be learned as part of building a NN in course 1.

By Ananthapadmanabhan K

Oct 1, 2017

Very good course. But unlike the previous one, I got the feeling we covered too much ground before the quizes. Perhaps having multiple quizes could make the learning stick a bit more.

Also, the intermediate verification of the programming assignment (tensorflow) was a little less satisfying than before because the intermediate "expected results" were mostly results of variable initialization or allocation. This is understandable because the "run" doesnt happen until a few steps later. But overall, the previous exercises were more rewarding even leading up to the big final "run everything together" part.

By Christian M N

May 8, 2020

Videos are very good, can't really criticize much there. The assignments are WAY too easy though. There's really no reason why they can't just let you implement more of the boiler/helper code, since everything is already auto graded. I think that guiding the user with the notebook structure and providing a description for what each function should do and the output shape should be more than enough for any user to do these assignments. As it is, these assignments don't really provide any assurance that that you understand the material, since the answer is always in the description.

By Abhishek R

Jul 14, 2020

It's a very useful course. The course material is well spaces out and encompasses the basics requires to utilise an algorithm. The programming exercises are easy to implement and rather rewarding when completed. Perhaps the only issue I had with the course was that it only taught enough tensorflow to allow us to implement the functions necessary for that particular problem. A more general understanding could have helped. Also, it, like most other ML/DL courses, never addresses the issue of actually accumulating or acquiring a dataset, which I find to be rather important to learn.

By Eric W

Feb 11, 2021

Hi,

just a brief feedback from my side. As a beginner in Deep Learning and Machine Learning in general the course was really the teaching style I was looking for, quite close to what I was and am used to at the University. One particular thing that was not so easy to come to terms with was the switch from mathematical notation in the slides to the Python expressions used in the assignments. Of course it was possible to learn them after some time, but I felt the learning curve a bit steep at that point assuming one has barely practical experience with it.

Anyway, a great course!

By Daniel C K

Sep 9, 2017

Good overview of techniques to use in deep learning. I especially appreciated the introduction to Tensorflow. The projects felt like they held your hand a bit too much. I'd like to see more rigorous assignments in the future. The Tensorflow tutorial felt unpolished. It contained a few errata and the tutorials were sometimes vague about how to use the syntax. I ended up having to look up the documentation for a few of the commands to understand how to use them. That's a minor issue for me, I have experience working in python, but for other learners it might be an issue.

By Andrew F

Feb 11, 2019

I enjoyed everything in this course. As always, the concepts and motivations for the various deep learning techniques are very well explained. I feel like I could have a conversation with members of the industry and not come across as a total novice (although I am).

The only real issue I had was regarding Tensorflow. I felt more time should be spent on Tensorflow before assigning homework on it. I believe in a certain amount of learning though struggle when it comes to programming, but I think the foundations for tensorflow were lacking to sufficiently struggle through.

By Rajesh R

Oct 27, 2017

A continuation of the Deep Learning specialization, this course actually teaches many of the latest ideas in hyperparameter optimization and builds up the ideas nicely. Concepts like EWMA, RMSProp, Momentum and Adam were well explained and logically introduced. One gripe I have is Tensorflow - despite not liking the framework much (primarily because of its terrible error messages), I found that the foundational learning for key concepts in TF were not well introduced. There was only one video which served as a Tensorflow introduction. Other than this, the course was great.

By Martin S

May 23, 2021

It's really great to the get the key insights from someone as Andrew Ng. I read some materials before in books, tutorials and other sources. But this course actually put it all into perspective and gave many aspects context which I was missing before. Thank you Andrew. If there would be one thing I would suggest to improve its the assignments. I missed using the batch norm and seeing its effect and also to actually improve the parameters of some DNN in some kind of search as was explained. But I understand that I might have been too much or rather too demanding.

By Nachiketa M

Sep 29, 2017

TensorFlow uses slightly different terminology. That took some time to get used to. Also, because of so many algorithms, parameters, it becomes difficult to remember everything. It would be nice, if we could search for terms and either get to the point in video where the terms are mentioned and explained or results contain sections of slides as presented in the video. If this kind of search works across all the courses in a specialization, that would also be great. Overall, Andrew Ng makes it really easy to understand and I am glad I am making progress.

By Deepak K G S

Oct 6, 2019

Very good presentation material and deep intuitive explanation by Professor Andrew NG . Quizzes are structured in a way that it will test all the sections you have learned in that week.Programming assignments will provide an idea on how to approach each of the problems.In addition to taking this course,I would also recommend you to try and code using the basic code structure presented in the lecture and assignments to build upon and see what works and what does not in the problems.Only then,you can see the benefit of what is taught in this lecture.

By Ravi R

Oct 10, 2020

Very good course for those who wants to start their career in Deep learning. One thing which disappointed me was programming assignments. Some functions and libraries of python and Tensorflow used in assignments are not well explained for students. How can a student understand if he wants to write code from scratch to develop an algorithm. Only small function or some logics are allowed to write by students in assignments which is not enough to build confidence. Many libraries imported are not explained. Theory part of course was excellent.

By Lucas S

Oct 14, 2018

Theoretical part amazing, explanations very comprehensive. I think is the best course on deep learning right now. The one thing i would like to see improved is the structural choice for the practical part: this concept of functions that already built in and we come in and fill in the code is ok, and it helps, but i think there should be some kind of path where we can build everything from scratch and you guys could provide the answers as something we can check to see if we got it right...i don't maybe could help, but overall amzing course.

By Markus B

Sep 7, 2017

Pro: The course content is well explained and the examples are usually understandable. There are some well explained programming exercises that allow you to get in touch with the "machine room".

Con: Not enough programming exercises to explain all concepts and also the programming exercises sometimes boil down to copy&pasting some code from the instructions. Furthermore, I would expect that this couse with "intermediate" difficulty would allow you to really write code from scratch at some point instead of filling in "Jupiter" notebooks.

By Mijael M

Mar 26, 2018

The content is great as usual, but I have two minor complaints: the quality of prof. Ng's handwriting went down, which sometimes makes it more difficult to follow the explanation, or taking notes; the second one is that the error reporting for the grader on the programming exercises is not very user-friendly. I had to try a couple times on weeks 2 and 3 not because my code was wrong at all, but because the grader was finicky. Due to the honor code, I believe I won't be able to post more details here, but check on the forums.

By Aaron L

Nov 25, 2017

The content is great, let me say that first and foremost. I feel like some small improvements would be that there were some typos in the notebooks, but not big deal. Helps one learn about debugging and think independently.

About the TensorFlow section of week 3. It was a pretty deep dive into TensorFlow, and I ended up going to the documentation a lot. Maybe some background on the framework would help. I will probably next go through the TensorFlow "getting started" tutorial, to better understand it.

By Eduardo M

Oct 29, 2018

The course is pretty good. I just feel notation used by Andrew is quite confusing for people with matrix algebra and matrix calculus. I understand the course is intended for people with different knowledge leveles but generalizing notation with matrix algebra could be save time for students.

There are some little bugs on the last Jupyter implementation with TensorFlow. Nothing too serious but reading the forums I noticed bugs were reported one year ago and nothing has been done to fix them.