43
SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS

SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

  • Upload
    others

  • View
    2

  • Download
    0

Embed Size (px)

Citation preview

Page 1: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

SDS PODCAST

EPISODE 317:

A DEEP DIVE INTO

NEURAL NETS

Page 2: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Kirill Eremenko: This is episode number 317 with aspiring data

scientist, Edis Gonuler.

Kirill Eremenko: Welcome to the SuperDataScience podcast. My name

is Kirill Eremenko, Data Science Coach and Lifestyle

Entrepreneur. Each week we bring you inspiring

people and ideas to help you build your successful

career in data science. Thanks for being here today,

and now let's make the complex simple.

Kirill Eremenko: This episode is brought to you by my very own book,

Confident Data Skills. This is not your average data

science book. This is a holistic view of data science

with lots of practical applications.

Kirill Eremenko: The whole five steps of the data science process are

covered from asking the question to data preparation,

to analysis, to visualization, and presentation. Plus,

you get career tips ranging from how to approach

interviews, get mentors and master soft skills in the

workplace.

Kirill Eremenko: This book contains over 18 case studies of real world

applications of data science. It comes off with

algorithms such as Random Forest, K Nearest

Neighbors, Naive Bayes, Logistic Regression, K-means

Clustering, Thompson sampling, and more.

Kirill Eremenko: However, the best part is yet to come. The best part is

that this book has absolutely zero code. So, how can a

data science book have zero code? Well, easy. We focus

on the intuition behind the data science algorithms, so

you actually understand them, so you feel them

through, and the practical applications. You get plenty

Page 3: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

of case studies, plenty of examples of them being

applied.

Kirill Eremenko: And the code is something that you can pick up very

easily once you understand how these things work.

And the benefit of that is that you don't have to sit in

front of a computer to read this book. You can read

this book on a train, on a plane, on a park bench, in

your bed before going to sleep. It's that simple even

though it covers very interesting and sometimes

advanced topics at the same time.

Kirill Eremenko: And check this out. I'm very proud to announce that

we have dozens of five star reviews on Amazon and

Goodreads. This book is even used at UCSD,

University of California San Diego to teach one of their

data science courses. So, if you pick up Confident

Data Skills, you'll be in good company.

Kirill Eremenko: So, to sum up, if you're looking for an exciting and

thought provoking book on data science, you can get

your copy of Confident Data Skills today on Amazon.

It's a purple book. It's hard to miss. And once you get

your copy on Amazon, make sure to head on over to

www.confidentdataskills.com where you can redeem

some additional bonuses and goodies just for buying

the book.

Kirill Eremenko: Make sure not to forget that step is absolutely free. It's

included with your purchase of the book, but you do

need to let us know that you bought it. So, once again,

the book is called Confident Data Skills and the

website is confidentdataskills.com. Thanks for

checking it out, and I'm sure you'll enjoy.

Page 4: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Kirill Eremenko: Welcome back to the SuperDataScience podcast, ladies

and gentlemen. Super pumped to have you back here

on the show. What an episode I have prepared for you

today. I literally just got off the phone with Edis just

under an hour ago, and Edis is a very inspiring young

man who is breaking his way through into the field of

data science. What you need to know about Edis is

that he's already participated in a couple of Kaggle

competitions. In one of them, he actually finished in

the top 18%. He knows deeply about neural nets. He

actually programs them for his Kaggle competitions, as

you'll find out for the podcast.

Kirill Eremenko: Something else very impressive I know is that he

creates neural networks from scratch. He doesn't use

PyTorch or TensorFlow, just because he wants to learn

how neural networks work. It takes him a week or so

to program a neural network, but he really dives deep

into it. He's started an artificial intelligence club in

New York, and lots and lots of very cool things are

happening in his life. He got into the field of data

science just under two years ago, just two years ago or

so.

Kirill Eremenko: But the most surprising thing you'll find is that Edis is

only 15 years old. That's right. 15 years old, and he's

already doing these massive crazy cool things. In this

podcast, he talks about Generation Z, and how they

use artificial intelligence, and often don't even realize

that there is artificial intelligence behind the products

that they're using. We talked about Edis's path, and

how he got into the space of data science, why, and

how he continued progressing to artificial intelligence.

Page 5: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Kirill Eremenko: We talked about his Kaggle competitions. Edis shared

how he codes models from scratch, and why. We

talked a lot about neural nets. We actually had a very

deep dive into neural network. If you don't yet

understand how they work, or you want to refresh on

your knowledge on neural networks, we discussed

that. We talked about the neurons, layers, weights,

activation functions, normalization, standardization of

data, min/max functions, quite a lot of things. That

was a really cool discussion. For me personally, it was

great to revise some of these things, and refresh them

in my own knowledge. I'm sure you're going to enjoy all

this.

Kirill Eremenko: Plus, in addition to that, we talked about feature

engineering, and how he used this technique in one of

his Kaggle competitions. Plus, we took it to the next

level, and Edis told us how he built an ensemble of

neural nets. That was the first for me to hear, that

somebody's building an ensemble, not of just decision

trees to build a random forest, but ensemble of neural

nets. That was a very exciting conversation, and you'll

learn about how that works, and how you can build

your ensembles of neural nets, and what the

advantages of that are.

Kirill Eremenko: Finally, we talked about the motivation, what keeps

Edis going. All in all, a very exciting inspiring episode.

Can't wait for you to check it out. Without further ado,

I bring to you aspiring data scientist, Edis Gonuler.

Kirill Eremenko: Welcome back to the SuperDataScience podcast, ladies

and gentlemen. Super excited to have you back here

on the show. Today, we got a very special guest calling

Page 6: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

in from New York. Edis Gonuler. Edis, how're you

going?

Edis Gonuler: I'm going good. Thank you for having me here.

Kirill Eremenko: Super excited to have you. Super excited. Since we

caught up at DataScienceGO in San Diego, how's your

life been? What's been happening? It's been a few

months since then. Right?

Edis Gonuler: Yeah, yeah. It's been a few months, and it's been great.

I'm really motivated to create data science projects to

explore different fields. It's just been awesome.

Kirill Eremenko: That's really cool. What I was wondering about is

what's got you to come to DataScienceGO in the first

place? How did you hear about the conference?

Edis Gonuler: Well, I listen to a lot of your courses on Udemy, and

that's basically a lot of the content that I studied. I saw

it in the email that you sent me, or someone sent me,

and I knew I had to be there. I knew it was going to be

a great experience, and that's how I went there. I told

my dad, and I said, "Dad, I have to go here. I want to

go," and he said, "Sure."

Kirill Eremenko: Wow.

Edis Gonuler: That's how it all happened.

Kirill Eremenko: Interesting, because I thought your dad brought you

there. It was the other way around. You encouraged

your-

Edis Gonuler: [crosstalk 00:07:22].

Page 7: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Kirill Eremenko: That's so cool. You live in New York, so you flew all the

way from New York for the event.

Edis Gonuler: Yeah.

Kirill Eremenko: That's so cool.

Edis Gonuler: I went to San Diego. It was a great experience.

Kirill Eremenko: That's so cool. What fascinated me the most is, you

probably noticed, you were by far the youngest person

there. How old are you?

Edis Gonuler: I'm 15 years old.

Kirill Eremenko: 15 years old, and you're at a data science conference.

Well, I got to caveat that. I did see one kid. Did you see

that kid? Like five or six years old.

Edis Gonuler: Yeah, I saw him. [inaudible 00:07:55]. Yeah.

Kirill Eremenko: He was there, and his mom was like, "Oh, you got to

learn data science. You got to learn mathematics." She

was really pushing him, five years old. Apart from that

kid, you were by far the youngest, and you were

getting a lot of value out of it. You were asking the

right questions. You even came to the workshops. Was

that right?

Edis Gonuler: Yes, I did. I attended your workshop as well as

creating R websites using Shiny, and they were really

valuable experiences.

Kirill Eremenko: Fantastic. Well, so what I want to ... Of course, I want

to have a fantastic chat, and see how I can help you on

this podcast, but I also want to see how you can

inspire our listeners because at such a young age,

Page 8: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

you've already established for yourself that, "Hey, this

is the path I want to take." I personally have never met

anybody this young who is so far advanced in the

space of data science and artificial intelligence. Yeah,

let's kick things off. Why and how did you decide to get

into data science?

Edis Gonuler: My dad's a software engineer, so he already got me

into technology. Since I was very young, I've always

had a computer. When I was really young, I was

watching videos, exploring the internet. Then my dad,

he was talking to me about creating websites, and how

he'd create websites. One night, I remember we were

going over something, how websites work, and went

into AI, and explained what AI is for me. That night, I

just sat on my desk for 30 minutes, and I was like,

"Wow, this is so cool." I was just like, "Oh, my God.

Oh, my God."

Edis Gonuler: I started researching it, and I found a lot of great

courses. I started watching them, and I started

creating projects, and that's how I learned. That's how

everything started.

Kirill Eremenko: Okay, but you don't have AI at school, right?

Edis Gonuler: No. We do have computer science courses in school,

where they teach you a little bit HTML and CSS. But

we don't have a strictly AI class.

Kirill Eremenko: Okay. Was it hard to learn how to create AI?

Edis Gonuler: Well, there were some challenges, of course. There's a

lot of content online, which is really good. But also

could be a really bad thing, because not knowing

Page 9: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

which content to pick, and you could waste time trying

to pick content. But I just started on Udemy, and

listened to a few of your courses. They got me super

pumped up, and I just started exploring, and creating

projects, and learning from there.

Kirill Eremenko: Okay, got you. What is it that your dad said, do you

remember, that got you so inspired?

Edis Gonuler: I think he was explaining how he can predict what

movies users would like based on previous data, and I

was just, especially about that example, I was so

interested in how it worked. Yeah.

Kirill Eremenko: Okay, okay. Have you used Netflix yourself? Was it

cool to realize that that's happening in the

background?

Edis Gonuler: Definitely, definitely. When I learned what it is, I

realized that a lot of things in life, a lot of different

products like Netflix, self-driving cars, you actually use

AI. I've always heard the term AI, but I didn't really

know what it means. It was just like a light bulb going

off in my head.

Kirill Eremenko: Fascinating. I've never thought about this, but from

what you're saying, I'm gathering that we talk a lot

about millennials. I'm a millennial. If you're 16, then

you would probably be a Generation Z, because

millennial's define as someone who was, I think, 16

years old, at least 16 years old when the year 2000

came around. But you're already born in 2003. So

you're Gen Z. We talk a lot about Generation Z, and

how there is, I'm not talking in particular about you,

but in general, there's this addiction to social media,

Page 10: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

mobile phones, and Generation Z people are very good

with technology and so on. But it's interesting to hear

that until recently, you actually didn't know how

artificial intelligence works, even though you use it to

your advantage pretty much daily.

Edis Gonuler: Yeah, definitely. A lot of things that we use, and

something as basic as Siri, we don't realize that there's

actually AI behind it. That was just fascinating to me

when I first learned AI. When I was first watching

courses, I was like, "Oh, I use this every day, almost,"

and it just made me more motivated to learn it, and to

progress.

Kirill Eremenko: Okay. Well, tell me, how long ago was this

conversation with your dad?

Edis Gonuler: This was about two years ago when I was in eighth

grade, I believe.

Kirill Eremenko: Oh, my gosh. Wow, your dad's a legend. That's so cool.

Huge props to your dad. Okay, so you were 14, you

had this conversation. I'm assuming you didn't know

Python back then.

Edis Gonuler: I was actually 13.

Kirill Eremenko: 13. Wow.

Edis Gonuler: Yeah. Yeah, back then I didn't know Python. I did have

a little bit of experience with HTML, just basics. But

after that conversation, after I started researching

online, I realized that it was either Python or R. I

looked at a few of the examples, and Python seemed

more like pseudocode or readable code. I decided to

just pick Python and go with it. From there, I started

Page 11: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

learning Python, the basics, variables, loops, and it

just got me more and more fascinated as I went.

Kirill Eremenko: Cool. How long did it take you to learn Python?

Edis Gonuler: It took me about six months, I would say, because I

was at school meanwhile, so I was trying to balance

school, and I was trying to learn.

Kirill Eremenko: Six months to get to what level?

Edis Gonuler: Six months to get to an intermediate level. I knew most

of the concepts in Python. That's when I started

learning AI.

Kirill Eremenko: Okay. All right. Which course in AI did you take?

Edis Gonuler: The first course that I actually took was your course

on Udemy. It was Machine Learning A-Z.

Kirill Eremenko: That's a big course.

Edis Gonuler: Yeah. I finished it in about a month, because I just-

Kirill Eremenko: No way.

Edis Gonuler: ... countless amount of nights, countless amount of

nights. I slept really, really late. I woke up early in the

mornings before school just to watch a few lectures.

Even on my commute to school, I used to watch the

lectures, and I was always thinking about, even in

school, how would I solve this problem? Or how would

I do this?

Kirill Eremenko: Wow. That is crazy. That's like a 40-hour long course

with over 200, or even 300 tutorials. That's insane to

do it in a month.

Page 12: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Edis Gonuler: [crosstalk 00:14:45] a lot content.

Kirill Eremenko: Well, the best ones I heard was people doing it in two

months, and average is about three months, I think.

You did it in one month. Crazy, man. Your knowledge

of Python was sufficient to take that course by then.

Edis Gonuler: Yeah, definitely, it was sufficient enough. Also, in that

course, it's also written in R, half bits in Python, half

bits in R, so I also explored onto R to learn some of R

at that time.

Kirill Eremenko: Okay, okay. Machine Learning A-Z, cool. Machine

learning is indeed a type or form of artificial

intelligence. Did you ever go deeper? Did you go into

reinforcement learning on those types of concepts?

Edis Gonuler: Yeah, definitely. Machine learning was like a gateway.

It was really, really interesting. But when I saw deep

learning, and reinforcement learning, I got even more

interested and motivated to learn all this stuff. That's

when I started buying the books, the heavy math

books, because I realized that a lot of math needs to be

learned in order to actually grasp these concepts. I

started reading book after book, learning all these

concepts.

Kirill Eremenko: Okay, got you. You're probably way ahead of your

class in terms of mathematics.

Edis Gonuler: Well, the thing is, in school when they teach you math,

you don't really understand why they teach it to you.

That's why I really love data science and AI because all

the things, all the math that you learn in school, you

Page 13: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

can apply it, and it's just cool to see how it all clicks

into place.

Kirill Eremenko: Okay, okay. Very cool. All right. You learned the

Python, you did the course, you learned some

mathematics. Then what? When did you start doing

your first projects? When did you actually feel, I don't

know, a spark of creativity towards artificial

intelligence?

Edis Gonuler: Yes, so that was about last year when I was in ninth

grade. I started with Kaggle. I looked at a few

competitions, and I decided to join it. My first attempt

wasn't very successful. But I learned a lot, learned a

lot of different cool things like feature engineering is

very important. Definitely analyzing the data before

actually putting it inside a model and training it, it's

really important. After that, I joined a few other

competitions.

Edis Gonuler: The last competition, my most recent competition was

the 1C competition. It's a rushed software company,

where you had to predict how many of each item

would be sold in the next month. It was a really good

experience for me. I learned a lot of different cool

things. At the end, I got into the top 18% using

ensemble methods.

Kirill Eremenko: That's very impressive. To give us a sense of how big

your competition was, how many people, 18% out of

how many?

Edis Gonuler: I don't want to lie, but about 5,000, I believe, 4,500.

Somewhere over there.

Page 14: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Kirill Eremenko: That's very good. That's very good result.

Edis Gonuler: Yeah. I try to improve my results, and I try to join

more competitions because creating projects and

experience is the most valuable thing, and it's what

makes you learn. At one point, after watching all these

courses, and reading all these books, I realized that I

was learning, but because I wasn't applying it, I was

forgetting the stuff that I already learned. I just

decided I'm going to stop the books, and I'm going to

actually start creating projects. That helped me a lot.

Kirill Eremenko: Okay, got you. Let's go into that a little bit. I love

talking about Kaggle competitions. Top 18% in this 1S

company competition to predict total sales for

products in the stores. Plus, you did some

competitions before that on Kaggle, a few. You

mentioned ensemble methods. That got you quite far.

Tell us, if you don't mind sharing, exactly what

ensemble method did you use, and why, and how did

you go about getting to this solution?

Edis Gonuler: Okay, yeah. I looked at my data, and before using

ensemble methods, I tried neural networks, I tried

SVMs. I realized that it wasn't getting the result that I

expected, and the features that engineered weren't as

useful in these models. I was just like, "Let me just try

an ensemble method." I think I used a few neural

networks, and on top of that, put linear regression. I

think that's what got me the top 18%.

Kirill Eremenko: Wow. You used an ensemble of neural networks.

Page 15: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Edis Gonuler: Yeah. Neural networks and I think I also used SVM.

I'm not completely sure. But yeah, [crosstalk

00:19:43].

Kirill Eremenko: That's crazy. When people say ensemble methods, it's

usually a random forest, or something more basic.

Edis Gonuler: Yeah.

Kirill Eremenko: But you used an ensemble of neural networks. That's

a good next level. Okay. Tell me then, you used this

ensemble method, and how, because the way you say

it is like, "Oh, I tried SVM, I tried neural nets, I tried

linear regression, I tried anything else," how difficult is

it to put together a model? How long does it take you

to code one of those from scratch?

Edis Gonuler: What I try to do in projects, the whole point of doing

projects for me is to learn. It's not really to get into the

top 18%, or the top 5%, it's to learn. I usually code

them out from scratch, not using any libraries, just to

understand the math behind it.

Kirill Eremenko: Wow

Edis Gonuler: Usually, some of like linear regression, and multiple

linear regression take me maybe a day to train, and to

... But some of the more complex methods like neural

networks, building it from scratch could take me up to

a week just to build it, because I also have a lot of

different things going on. I don't really have that much

time to code.

Kirill Eremenko: Hold on, hold on. Did you build neural networks

without TensorFlow or PyTorch?

Page 16: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Edis Gonuler: Yes. I did it just to understand the math behind it.

[crosstalk 00:21:06].

Kirill Eremenko: Wow, that's crazy, man. That is insane. You build a

neural network without using existing libraries. That is

definitely going to get you to learn. I see your point.

You're going to know them so well after that. How do

you know that you didn't make any mistakes along the

way? There's so much math involved.

Edis Gonuler: Well, there is so much math. Of course, I didn't do this

on top my head, I researched online to see how it

would work, and then the formulas in the

mathematical aspect. I just kept on trying and trying

and trying, and then I compared from one of the

libraries, I believe from TensorFlow, the neural

network. I imported it, and also my neural network,

and it was about the same [inaudible 00:21:55]. It gave

about the same loss. I realized that the one I created

was pretty good, and it worked.

Kirill Eremenko: Okay. Wow. Very cool. All right, so you built your own

neural, and you said it took you about a week to build

a neural network from scratch.

Edis Gonuler: Yeah, about a week.

Kirill Eremenko: Interesting, because when asked you the question

from scratch, I actually thought to myself, "I shouldn't

have said from scratch because you probably were

using libraries, drag and drop, putting things

together." Because if you use TensorFlow or PyTorch,

it's going to take like two hours to build a neural

network. Maximum. All right, it's more about the

architecture that you decide upon.

Page 17: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Kirill Eremenko: Yeah, how do you go about that? How did you go

about experimenting with the neural net architecture?

What insights can you share? What did you learn for

that specific challenge about sales prediction?

Edis Gonuler: Yes. In the neural networks, online and in the books,

they usually recommend that in the middle of the

neural network, to put a lot of neurons, and then

towards the ends, not to put a lot of neurons. I realized

that to be true. But what I did in the beginning was I

just put a bunch of neurons, and I just tested it out.

The result that I think I got from that was, if you put

neurons in the first few layers, it's not as good as

putting them in the last few layers. In the beginning of

your neural network, try to keep it simple, and

towards the end, try to amp up the amount of

neurons.

Kirill Eremenko: Okay. Why do you think that is, because you coded

these from scratch, you know the math, you maybe

have a better sense about what's going on in there

than most of us. What would you say are your

comments? Why do you think you're seeing this

inside?

Edis Gonuler: Well, to be honest with you, I'm not 100% sure. But I

think that in the beginning, since the data is

completely ... it's fresh, nothing has been applied to it,

it needs to be basic, and it needs to just have, just

each neuron has to have a specific weight. Then

towards the end, as the numbers get all jumbled up, it

doesn't really matter as much. I think you can use

more weights there, and then to get an output. This is

my experience, and this is how it worked for me. I'm

Page 18: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

not sure how it would be for other types of neural

networks. But in my experience using this data set

from this Kaggle competition just seemed to work

pretty well.

Kirill Eremenko: Okay. What activation function did you use in your

neural net?

Edis Gonuler: I think I used ReLU. I'm not sure if I'm pronouncing

that right. But also I tried a lot of different activation

functions. I believe I use Adam, and I've realized that

ReLU was the easiest one to use, and it got me the

most effective results.

Kirill Eremenko: Okay. All right. You know what, I got an interesting

idea. I love doing these things, to recap my own

knowledge and solidify it, but also maybe somebody

listening to this will find it useful. Let's try together to

explain neural nets in a couple of minutes. Obviously,

we don't have visual aids because this is a podcast,

but it's going to be tough. But let's see if we can do it.

Neural net, how would you summarize it up? What is a

neural net?

Edis Gonuler: I would summarize it like this, I would say you're

trying to simulate how our brain works using a

machine. In your brain, you would have neurons. You

would get an input, and then that results in output.

Famous example for this is when you touch a hot pot,

you move your hand away. You're getting input that

it's hot, and your neurons are saying, "Well, you're

going to burn, so move your hand away." I think that

neural network is very similar to this. You get an

output ... An input. Sorry. An input, and it goes

Page 19: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

through all these neurons, and then you get an output

at the end. This is how I would probably sum it up.

There's a lot more to it, but in a few sentences.

Kirill Eremenko: That's a good start, it's a good summary. Now, let's

unwrap it a little bit. I'm going to describe a bit how it

works, and then you can add your comments

[inaudible 00:26:13]. Sounds good?

Edis Gonuler: Sure. Yeah.

Kirill Eremenko: Okay. Everybody can probably imagine, everybody's

already seen, I'm going to assume here that

everybody's already seen an basic image of a neural

net. You have these circles, vertically aligned, that's

your first input layer. It can be three, five, can be 100,

can be 1,000. Those are where your inputs come in.

Then, they're all interconnected to the next layer, next

vertical aligned layer of these circles, which are

neurons. It can be more circles, or less than in the

first layer, and so on.

Kirill Eremenko: All these layers are connected to the next one, to the

next one, to the next one, and in the end, you have one

output neuron, or maybe even several output neurons

in the output layer. They're connected, and what

happens is your information, just imagine, it's coming

in from the left into these input neurons. It has to be

obviously, any graphics, or any text has to be all

changed into numbers, as the numbers have to be

normalized, and so on. Basically, your inputs, which

can be different columns in your data sets as well,

they're any variables are put into input layer, then

what happens is, each of these, every next neuron in

Page 20: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

the next layer, let's pick a random neuron, is

connected, let's say to all of the neurons in the

previous layer with these lines. These lines are called

Synapses. Every synapse has a weight assigned to it.

Kirill Eremenko: The neuron gets assigned to value, which is equal to

the weighted sum of the input values multiplied by the

weights of the synapses. It's basically as simple as

that. When I learned this for the first time, I was so

shocked that it's just as easy as a weighted sum, all of

your input values in that neuron. Then, an activation

function's applied on top of that, which we'll talk about

in a bit. Then the same thing happens for all the

neurons in that hidden layer. Then again, this process

repeats for next layer, next layer there.

Kirill Eremenko: Basically, all the time, all that's happening is you're

adding up the values that you have in your current

layer, multiplied by the weights in the synapse to get

them a value in the associated neuron. You're

calculating that for all the neurons, and you're going

through the whole neural network. That process's

called Forward Propagation goes from left to right.

Does that sound about right?

Edis Gonuler: Yeah. Yeah, that's a great way to sum it up. Like you

said, it's as basic as that. It's nothing really complex.

It's basically like linear regression is a really good

foundation for that. That's basically what linear

progression does, but a neural network just does it in

a much wider scale, in a much larger scale.

Kirill Eremenko: Yeah, that's a good way of putting it as well. Okay,

cool. That's how they work. Once your neural network

Page 21: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

is trained, and the weights are defined, that's how

exactly it works. Then, a question for you, Edis, where

do the weights come from?

Edis Gonuler: Okay, so the weights are first initialized. And then

after that, [crosstalk 00:29:05].

Kirill Eremenko: What does that mean? Initial.

Edis Gonuler: They're initialized. They have a value. You first set

them all to zero or one. When the numbers pass

through as they go through the neural network, they

get multiplied by one, or they get multiplied by zero.

Then you get an output from those inputs. After that-

Kirill Eremenko: I would add a small correction there. It's usually a

value close to zero. If they're all zero, then it's just

going to be ... not going to really work.

Edis Gonuler: Yeah. It's going to be zero. Then after that, after you

get the output, it compares it to the output that it's

supposed to get. When you're training the model, it

compares to the output that you're supposed to get. It

looks at the difference in the last function. Based on

that, it changes the weights. It either makes them

more, or makes them less, and it tries to make this

neural networks get an output that's as close as the

real value. This is just a summary. There's way more

to this than I'm explaining. But this is basically just ...

Yeah.

Kirill Eremenko: Yeah, exactly. In forward propagation, the data goes

from, or the information goes from left to right, feeds

forward through a network. Then, as you said, in the

training data set, you actually ... because by definition,

Page 22: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

in the training data set, you already have the outputs

that you're supposed to get. If you're training a neural

network whether to distinguish between a dog and a

cat, you already have like 10,000 images, which are

labeled dog and cat. You have the answer. You

compare it to the answer, and depending on your loss

function, or your error, you do back propagation.

Kirill Eremenko: Then, you propagate the error back through a

network. Basically, what that means is that the

network automatically just slightly permutates those

weights, so changes them a little bit to reduced error,

and to reduce the error. There's processes behind that

such as Gradient Descents, Stochastic Gradient

Descent, Batch Gradient Descent, and so on, which

we're not going to go into. But basically, there's ways

that the neural network can slightly adjust the weights

to see if it can get the answer better closer to the

result. That's why you need a lot of data, right, so that

you can do this thousands of times before you get

those final weights that work well for your problem.

Edis Gonuler: Yeah, yeah, definitely. This was a good summary of it.

It's not very complex, but I feel like it could be, and

there's a lot of math behind it, which it's good to

understand because that's a good foundation for it.

Kirill Eremenko: For sure. Look at us, we explained neural nets on the

podcast. The question is how useful that is. I wonder if

listeners got value out of that, but hopefully they did.

Edis Gonuler: Hopefully.

Kirill Eremenko: Hopefully. Activation functions are simply the function

that's applied to the output of the neurons. The input

Page 23: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

into a neuron is the input, so the input of the neuron

is whatever the values were in the previous neurons,

time's a weight, so weighted sum, and then the output

of neuron is that weighted sum, but we apply an

activation function. A very simple one is the threshold

function where if the weighted sum is less than zero,

then we turn that neuron off, and make it a zero. If it's

more than zero, then we turn that neuron on, and

make the output a one. That's a very simple activation

function called the Threshold Function. The one that-

Edis Gonuler: Yeah, that's [inaudible 00:32:23].

Kirill Eremenko: Yeah. Then the one that you use is called the ReLU, or

rectifier. How would you explain that one?

Edis Gonuler: Yeah. If you look at a graph basically below zero, it's a

straight line, and after zero, it's the line y=x. This is

used for predicting numbers, and not probability, so

not a percent. But it's used to predict numbers. It's

very useful in a lot of different cases, like in the

competition that I had, where you would predict the

number of items sold. This was the best. This was the

activation function that I found was most useful.

Kirill Eremenko: That's very accurate description. It's basically, it's kind

of like the threshold function, the very simple one that

we just talked about, but where below zero, the

neuron's off, above zero, the neuron is equal to one.

Here, it's like if your value is below zero, then the

neuron's off. But if your value is above zero, then your

value, your output of the neuron is equal to the value.

You maintain the value, don't change it to a simple

Page 24: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

one. Okay. What other activation function can you

think of?

Edis Gonuler: I can think of Adam. I haven't really used it. I've just

heard of it. I've heard a lot of good.

Kirill Eremenko: What about the Sigmoid Activation Function?

Edis Gonuler: Yeah, so the Sigmoid Activation Function is also

useful, but it's useful in different case. When you're

predicting probability, you can use a Sigmoid

Activation Function, and what it will do is you would

set a threshold, so below 0.5, so below 50%, or above

50%. If you're, for example, predicting if a cell is a

cancer cell or not, and you get a percent, you would

want to change it to either yes or no. You would say if

it's below 0.5, it's a no, if it's about 0.5, it's a yes.

That's basically at its core what the Sigmoid Function

is.

Kirill Eremenko: Okay. It's the same function that's used in logistic

regression.

Edis Gonuler: Yes. Yeah.

Kirill Eremenko: Okay, fantastic. That's just three activation functions.

I checked the Wikipedia page recently for activation

functions, there's like 30 of them, at least.

Edis Gonuler: There's a ton. Yeah.

Kirill Eremenko: It just shows that in neural nets, there's a lot of room

for creativity. As the architect of the neural network,

you can decide, how many neurons am I going to

have? How many layers am I going to have? And what

activation functions am I going to use, and lots of

Page 25: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

other things, but those are the main things that you

can tweak, and just design your own neural network. I

wanted to ask you, what do you like more? Do you like

the mathematics aspect, and coding the neural nets?

Or do you like the architectural aspect of the neural

networks?

Edis Gonuler: I think they tie in together, because when ... I like

personally the math aspect a little bit more, because

you understand how it works, and it's good to know

the math to actually build architecture, because when

you know the math, you know how it would work.

Based on that, you could predict even before training

your neural network, because neural network takes a

lot of time to train. If you don't have a good

architecture, you could waste hours of time, and not

have a good model. I think they tie in, but personally,

me, I like the math aspect a little bit more.

Kirill Eremenko: Okay. Got you. Got you. You also mentioned feature

engineering, right? That's a very important part for any

kind of machine learning, or artificial intelligence for

that matter.

Edis Gonuler: Definitely.

Kirill Eremenko: Tell us a bit about that. What is feature engineering?

What kind of cool features did you come up with for

this Kaggle challenge that you were in?

Edis Gonuler: Yeah. Feature engineering is creating a new feature

based on the data that you already have. One of the

features that I created for my competition, for the

competition I was in was, there was a bunch of items,

and I created a feature in CD. If the word CD wasn't

Page 26: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

the item name, then in the column is CD, it would be

yes or no. This helped me because I realized that CDs

were sold more than other products. This helped me to

just take them, and put them on a different side, and

say, "Okay, this is a CD, the way it's going to be a little

bit more. Or how am I going to ... It made me think,

"How am I going to use this to build a better neural

network, to build a better model?"

Kirill Eremenko: Very cool. Okay. That's an unusual way of engineering

a feature, so you're pulling name from text data, or

values from text data, creating a Boolean variable.

Very cool. Indeed, maybe some domain knowledge

about the company, the market that it's in, that 1S

company indeed sells a lot of CDs, because I think a

lot of them are used for corporate type of accounting

software and things like that. That's one feature. Did

you do engineering other features?

Edis Gonuler: The other features were just the means of the

columns, and the mode, and other stuff. It wasn't

major. This one was the most interesting one, and the

one that I realized was the most effective, I would say.

Kirill Eremenko: Okay. Okay, got you. What do you mean, mean of the

column? You take a mean of the column, and you

would create a feature that has the same value across

all of the rows?

Edis Gonuler: Yes. What I would do is I would take the mean, and I

would subtract it from the actual value, so the

difference between the mean and the actual number. I

realized that in some competition, this helped, because

in my previous competition, this was a useful feature.

Page 27: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

But in this competition, it wasn't as useful as it was in

my previous competitions. It varies from competition to

competition.

Kirill Eremenko: Okay, okay. Got you. Interesting. Feature engineering,

you'd think that you have all these inputs, you just

use them. Why is feature engineering so valuable?

Edis Gonuler: Feature engineering is important to bring out the best

in the data. Some of the features might not be useful

in the way that they're presented. But if you can

extract something from that data, most of the item

names wouldn't be used because they're text. It would

be hard to, if you can use NLP, but it would be harder

to translate them to numbers. But if I know that I can

pull CD from the item name into a separate column,

that would be way more useful than spending my time

changing the text into numbers. Just pulling,

extracting the parts that you need from data.

Kirill Eremenko: Okay. Okay, got you. Understood. How did you

normalize your data before feeding it into the neural

net? Did you use standardization, or did you use

min/max? What did you use?

Edis Gonuler: I tried it out a lot, and this was a few months ago. I'm

not completely sure, but I used min/max, I believe, in

my final submission, because standardization, I

believe it didn't work as well for me as min max, and

it's just based on the data, min/max, which is

basically taking the maximum of the column, and then

minimum of the column, and subtracting them. I

believe that worked better for me. Like I said, all these

things, like feature engineering the model, they all

Page 28: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

depend on the data. That's why exploring your data

and visualizing is very important thing.

Kirill Eremenko: Yeah. Okay. Yeah. We'll talk about that in a second.

Let's talk about for a second as well, about

standardization, or normalization of data. Why do you

need to normalize variables before feeding them into a

neural network? What would your one paragraph

summary, or comment on that be?

Edis Gonuler: Yeah, so normalization or standardization is very

important, because there's a lot of outliers that can be

outliers in data. Aside from outliers, some numbers,

like in one column, the numbers could be thousands.

It could be 100,000, and in the other column, it could

be one, could be a value of one. If, let's say in this

competition with the items, the amount sold last

month could be two, but the price sold could be 100.

When you put these numbers inside a neural network,

two and 100, one is obviously bigger than the other,

and the neural network could alibi saying that the

number that's bigger, it's stronger. That feature is

more important.

Kirill Eremenko: It's overwhelming.

Edis Gonuler: Yeah. To not have this problem, you standardize, and

you convert them into small numbers, usually between

-1 and one, or -two and two, and it solves this problem

of having a bias model.

Kirill Eremenko: Fantastic. Fantastic. Well, I love the description.

Indeed, you want to make sure the numbers are not

only ... You can't add kilometers and kilograms, for

instance. You want them to have the same unit of

Page 29: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

measurement, or no unit of measurement for that

matter, and you want them to be on the same scale,

like you said. If you have a one and a 200 value, or

100 value, then one will overwhelm the whole neural

network. One way to normalize the process of bringing

old values to a common scale is called normalization.

Kirill Eremenko: One process of doing that is called standardization,

where you subtract the mean of that variable across

the whole column. In every row, you subtract the

mean of that variable, and divide by the standard

deviation. That gives you a value, which is distributed

with a mean of zero, and a standard deviation of one.

Then that way, all of your columns become like that.

Or the other one that you said you used in your Kaggle

competition in the end was min/max, where you just

take ... Maybe you can describe this one better.

Edis Gonuler: Yeah. You take the value that you want to normalize,

and you subtract it from the mean. You got to subtract

the main from that value, and then you divide it over

the difference between the maximum and the

minimum value in that column.

Kirill Eremenko: Yeah. That gives you a number between zero and one

every time.

Edis Gonuler: Yeah.

Kirill Eremenko: Yeah. Well, yeah, naturally. That's another way of

normalizing variables. There we go described. I think

that's a very important concept in a lot of machine

learning, including deep learning to understand how

best to normalize. Even depending on what you

choose, you choose standardization, you choose

Page 30: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

min/max, or some other approach. That's going to give

you a different result, and that's what you saw in your

competitions, right?

Edis Gonuler: Yeah. You might see like one or two decimal places off.

It's not a big deal. But at the end, when it goes

through all these different models, those numbers,

even if it's a one, or a zero, or a -1, it makes a huge

difference at the end.

Kirill Eremenko: How would you describe that? Why does it make such

a big difference? Why does the type of normalization

that you choose play such a big role?

Edis Gonuler: Well, because there's a lot of these numbers, and data

sets normally are really big. When it goes into these

models, if a number is between zero and one, it would

get a different result than if it was between -1 and one.

When it goes through these models, something that is

not negative wouldn't make a huge effect.

Kirill Eremenko: Yeah, okay.

Edis Gonuler: It would make a different effect.

Kirill Eremenko: Yeah. Another point would probably be that some of

these activation functions like the ReLU are not

symmetrical around zero. If you have negative

numbers there, a lot of them will get dropped off along

the way. Interesting. Okay. You mentioned another

thing about neural nets is analyzing, or exploring your

data before the model, and how that helps. Tell us a

bit about that. Why can't I just take my challenge, and

jump straight into modeling, and creating these

Page 31: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

features, and building a neural net right away? What's

the benefits of exploring your data beforehand?

Edis Gonuler: Well, you would know which features have the biggest

correlation with the result you want to get, with the

output. You want to explore your data, and you want

to find these correlations, and you want to also see

what kind of data do you have? Do you have text? Do

you have numbers? Do you have categorical features?

Based on that, even when you're data pre-processing

in that stage, you want to know, what do you have to

do? What do you have to apply? That makes all the

more difference, just looking at your data and seeing

what is it? What is there?

Kirill Eremenko: Yeah, okay. When you say explore, what do you mean

by that?

Edis Gonuler: Just create graphs, create charts, create heat maps to

visualize the correlation between two features. There

are great libraries for these, like Matplotlib, Seaborn is

a really good one that I use. It helps you a lot in the

other steps in building your model, in pre-processing

your data because you get to see all your data at once.

Kirill Eremenko: Okay. All right, so you got to look for these correlations

beforehand, and then you know better which features

will likely be most important for your neural net, or

which features you can engineer that will be useful.

Tell me then, you said you use ensemble of neural

nets. How does that work? What were the differences?

Was it like the same neural network, a typical

ensemble method where you just feed in different

subsets of the data to the different neural nets, and

Page 32: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

then you average out their response? Or were the

neural nets themselves different to the ensemble?

Edis Gonuler: Yeah, like you said, there's two ways to do that. I

picked the first way that you said. What I did was I

created four or five different neural networks. I trained

them all with data, and then I used a linear regression.

I averaged them, and I use a linear regression to

predict the output, to pick the output. I think that

when you average them, it's really good because each

model can have a different bias towards a feature.

When you average them, it doesn't cancel out, but it

gives a new perspective. I mean, these are just

numbers. Your perspective sounds wrong, but that's

really what it is.

Kirill Eremenko: Interesting. Okay, so it was the same neural network,

a copy/paste five times of the same neural network.

Right?

Edis Gonuler: I did play around with some of the neurons, and the

way they were placed. But overall, it was about the

same type of neural network.

Kirill Eremenko: Okay. The same type neural network, but did you feed

in different subsets of your training data to them?

Edis Gonuler: Yes, I split it into four, or five different subsets, and I

gave each neural network one type, one of those

subsets, and changed it on that, and then validated it.

Kirill Eremenko: Interesting. How big was the training dataset?

Edis Gonuler: I can't remember off the top of my head, but it was

about 100,000.

Page 33: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Kirill Eremenko: Okay. So about 20,000 each.

Edis Gonuler: Yeah.

Kirill Eremenko: Were the predictions of the neural nets drastically

different?

Edis Gonuler: It weren't drastically different, but for some of the

items that are predicted, because these were items, so

they were in the one to 20 range, so they weren't

drastically different. But I would say averaging them,

just even if it's by a little bit, it improves the score, it

improves the loss function.

Kirill Eremenko: Yeah.

Edis Gonuler: It just helps a lot.

Kirill Eremenko: Ensebmle net is great in general. I find they're much

more stable than if you use a single method, because

as you said, because it just can be biased towards that

one feature in a certain direction. Whereas if you have

ensemble methods, you're in a way leveraging the law

of large numbers. It's like a democracy, you are less

likely to make an error. Even if one of them makes an

error, on average, they average out.

Edis Gonuler: Yeah.

Kirill Eremenko: That's very cool. Okay, so why do you choose that

approach rather than having five radically different

neural nets, and feeding all of them the same data?

Edis Gonuler: I thought that this would keep me more organized, and

I thought, when I thought about, I thought that this

would be the best way. Probably back then I didn't

know that there was another option, so I just decided

Page 34: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

to do a few neural networks with the data, and then

just average them. That would be the output, that

would be my prediction.

Kirill Eremenko: Yeah. Got you. That is more classic approach, because

that's the same approach you'd use for your decision

trees to create a random forest. That's all. Okay. What

did you use? I'm just curious, because an ensemble of

neural nets would require quite a bit of computing

power. Did you use Amazon, or did you use your own

computer? What did you use?

Edis Gonuler: I definitely did not use my own computer. That would

have be almost impossible. I remember training a few

on Kaggles kernels, on their notebooks. I believe I also

used Google's platform, Google Colab. I changed a few

there and here, and then at the end, I just wrote them

down to one file on my computer. I actually averaged

them out on my own computer.

Kirill Eremenko: Gotcha. Okay.

Edis Gonuler: [crosstalk 00:50:12].

Kirill Eremenko: Nice, very nice. Awesome. Well, very creative and a

non-unconventional approach to this problem. I

haven't heard somebody use an ensemble of neural

nets. But I guess we're entering the world of unlimited

computing power, where it's going to be easier and

easier to create crazy things like that. That's exciting.

Edis Gonuler: Yeah.

Kirill Eremenko: Thanks a lot for this discussion on neural nets. I think

that was very useful.

Page 35: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Edis Gonuler: Yeah, thank you.

Kirill Eremenko: Yeah, I haven't had this on the podcast yet where we

dove so deep into it. That was, for me, even great to

refresh on these things. I want to change gears a bit

and talk a bit about motivation. Where do you get the

motivation at 16 years old, right, to-

Edis Gonuler: 15.

Kirill Eremenko: 15. Sorry, man. At 15. It still doesn't sound right in my

head. At 15 years old, to go and keep plowing ahead,

waking up early, watching videos, coming home from

school, watching course videos again. Participating in

Kaggle competitions, going to conferences, reading

books about mathematics. Your friends are probably

out there, I don't know, playing sports, or partying, or

other things. How do you keep yourself motivated?

Edis Gonuler: Well, when you have fun, it's really easy to keep

yourself motivated. When I'm doing data science, I find

that playing sports doesn't make me happy. Partying

doesn't either, but when I'm doing these data science

projects, I see myself, I'm happy, I'm motivated, I'm

just there, and I'm willing to put in the time, and

effort, and I think that's the biggest thing. Motivation

defeats all the obstacles, and if you use motivation,

and if you put in the time, the effort, then all the

obstacles will turn into a valuable learning experience.

Yeah.

Kirill Eremenko: Okay. Okay. Very, very true. You do go outside

sometimes, right? Because it's-

Edis Gonuler: Yeah, yeah. Definitely.

Page 36: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Kirill Eremenko: Okay.

Edis Gonuler: Definitely.

Kirill Eremenko: All right. Okay, so you're just having fun. Did you start

having fun right away? How did you find that fun in

artificial intelligence, and neural networks, and

machine learning, because to be honest, a big part of it

is quite tedious data preparation, or even more coding

mathematics into code? How did you find the fun in it?

Edis Gonuler: Yes. When I first began, it wasn't as fun as it is now.

Like you said, the data preparation is a little bit

sluggish, and it's not as fun. But I always had the end

goal in my mind that if I learned this, and I get to the

models, and I get to the neural networks, then that's

when it starts to be cool. I was like, "Okay, it's coming,

it's coming. The next thing that I'm going to do, it's

going to be fun, it's going to be cool." When you put

that in your mind, you're willing to just learn these

things fast, and then get to the cool part, get to the

exciting part, and just do projects.

Kirill Eremenko: Yeah. You promised yourself it will be fun, and it

turned to be fun.

Edis Gonuler: Yeah, and it turned out to be actually more fun than I

expected. I was always saying, "This is the future. If I

want to learn something, I have to learn this." It just

kept pushing me. I kept pushing myself, and my

family was also very supportive of me. But I kept

pushing myself to learn, and to stay motivated, and to

put in the time and the effort. Now, I see that all the

work that I put in pays off, because now I'm actually

having fun. I'm creating really cool things.

Page 37: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Kirill Eremenko: That's very cool. At DataScienceGO, we spoke with you

about one question you asked me, and you put me on

the spot. I couldn't come up with an answer for you.

The question was, what is the next step for you? I was

lost for words, because at this age, you've already

progressed a lot, and at the same time, you still have

to finish school.

Edis Gonuler: Thank you.

Kirill Eremenko: You're welcome. Thank you coming to DataScienceGO,

and chatting to me. It was really inspiring. But

basically-

Edis Gonuler: [inaudible 00:54:25].

Kirill Eremenko: ... you've progressed so much ahead, and at the same

time, you still have to finish school, you have to finish

uni. I can't give you the normal career advice that I

provide people because those people are usually out of

university, and they are looking for careers. In your

case, as I understand, you probably still want to go to

university, and I connect you with a few people, or I

think you met a few people there at DataScienceGO,

who maybe helped with some advice. Have you found

an answer to that question over the past couple

months?

Edis Gonuler: Overall, I did. DataScienceGO, and thank you so much

helping and everyone that talked to me and gave me

advice, and I realized that I learned a few things

actually at DataScienceGO. The first one I've

mentioned over this podcast was doing projects.

Projects helped me learn, and I talked to a lot of people

there. They said, "Just do projects, keep learning, keep

Page 38: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

doing cool things." That's really been my goal. Ever

since I came home to New York from DataScienceGO, I

started an AI club at my school.

Kirill Eremenko: Wow. Congrats. That's awesome.

Edis Gonuler: Thank you. It's twice a month, and it started a few

weeks ago. Yesterday was our second meeting. We just

talk about AI and a little bit of teaching how AI works,

all the different things. Next week, hopefully, we're

going to start with our first algorithm, linear

regression, starting to predict house prices.

Kirill Eremenko: Nice. Very cool.

Edis Gonuler: Yeah.

Kirill Eremenko: Okay. How many people attend your meetup?

Edis Gonuler: About 25 people.

Kirill Eremenko: Wow.

Edis Gonuler: It's after school. Yeah. It's a great turnout. I didn't

expect that much. But a lot of people were interested,

and I called it AI Club, not that, Data Science Club,

because I realized if I call it Data Science Club, people

wouldn't know what that is. But if I called it AI club,

then there will be more people.

Kirill Eremenko: Nice. That's a great way to give back to your-

Edis Gonuler: [crosstalk 00:56:14].

Kirill Eremenko: ... community, to your peers to help others get in the

space. It's awesome.

Page 39: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Edis Gonuler: When I went to DataScienceGO, there was a huge

community of data scientists, and I realized that I

missed that when I come home. There aren't a lot of

people to talk to about all these things, like neural

networks, all the problems, and everything. That was

my motivation for starting AI Club. I realized that if

there was a community of people that want to learn

this, then there's always someone to talk to.

Kirill Eremenko: Yeah, wow. That's very true. It was cool. For me, every

time DataScienceGO is a really cool place to see

everybody get together, and how inspired, and creative,

and sharing everybody is with each other. I love that,

so I totally get your feeling. You want to recreate that

in your hometown, so you can have that not just once

a year, but every week, or every second week. Huge

congrats to you. That's a massive step forward.

Edis Gonuler: Thank you. Thank you.

Kirill Eremenko: Okay, so doing projects, and growing as part of a

community, helping your community grow. Those are

your next steps, and yeah, sounds really cool. Are you

excited?

Edis Gonuler: I'm very excited. I'm very excited for the future in AI

Club, progressing myself, doing projects. Yeah. I'm

really motivated, and I'm just ready to work.

Kirill Eremenko: Fantastic. Well, Edis I wish you the best of luck with

all these things. It sounds like such a great journey

[inaudible 00:57:46]. Yeah, and thanks so much for

coming on the podcast. This has been a great-

Edis Gonuler: Thanks for having me.

Page 40: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Kirill Eremenko: ... great, great episode. Before I let you go, what's the

best way for listeners to get in touch, contact you,

follow your career, and basically find you online?

Edis Gonuler: Yeah. My email, you can email me, my email's

[email protected]. I also have a LinkedIn. If you want

to text me or message me, you can do it from there.

Kirill Eremenko: Got you. Well, we'll include the LinkedIn in the show

notes for sure, and people can find you there. Highly

recommend to connect with Edis, and see how his

career progresses. Okay, and one final question for

today, you read a lot of books, so what's a book that

you can recommend to our listeners, something that's

inspired you to take your career further?

Edis Gonuler: I had two books in mind, and they both have two

different things. My favorite book for the mathematical

side of data science is The Hundred-page Machine

Learning Book. I think that it's really good, and from

scratch, you can learn a lot of the mathematic theory

behind it.

Kirill Eremenko: Very cool.

Edis Gonuler: Just an overall awesome book. The second book is

more of a ... It's a really good book as well. I think you

know it. It's Confident Data Skills.

Kirill Eremenko: Thanks, man.

Edis Gonuler: I read it. I picked it up at that DataScienceGO, and I

started reading it. I'm almost done with it, and it's just

a great overall book. There's a lot of great concepts,

and a lot of great charts and graphs that it just had

that I enjoy looking at, and just researching.

Page 41: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

Kirill Eremenko: Fantastic. Thanks for the shout out.

Edis Gonuler: [crosstalk 00:59:26].

Kirill Eremenko: That's really cool. Actually, as we speak, I have on my

computer open, the files for the second edition. I'm

working on the second edition.

Edis Gonuler: That's great.

Kirill Eremenko: Yeah.

Edis Gonuler: That's great.

Kirill Eremenko: I think it's already announced on Amazon, but it's

coming out mid next year, so very excited about that.

I'm glad.

Edis Gonuler: I'm excited too. You got me excited.

Kirill Eremenko: Man, I'm glad you're getting some value, and another

book too, The Hundred-page Machine Learning Book. I

haven't read it myself, but I'm sure there's a lot of

value in that. Once again, Edis, thanks for coming on

the show, and can't wait-

Edis Gonuler: Thank you for having me.

Kirill Eremenko: Yeah. My pleasure. My pleasure. Can't wait to catch

up again sometime soon and see where your career,

and your learning, your projects take you.

Edis Gonuler: Yeah, that would be great. Thank you.

Kirill Eremenko: All right. See you. There we have it, ladies and

gentlemen. That was Edis Gonuler, aspiring data

scientist. Somebody who is very young, but is already

powering through into the field of data science and

Page 42: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

artificial intelligence. I really hope you found this

episode as inspiring as I did. Make sure to connect

with Edis. I'm going to leave his LinkedIn on the show

notes. Personally from me, the most exciting and

inspiring part of this episode was probably, everything

was really cool. Probably, I'll say two things. The first

one was how he codes models from scratch is very

surprising. But the most inspiring was the fact that at

the start, he didn't find data science and AI fun, but

he promised himself they'll be fun, and they became

fun, and became even more fun than he thought they

would be.

Kirill Eremenko: I think that's a very cool psychological technique any

one of us could use in order to get more into

something that we really believe in, get into whether

it's data science, AI, into a hobby, an interest, a

profession. Promise yourself, it will be fun, and work

for it to become fun, and then assess in a bit of time. I

think that was a really cool tip. As usual, you can find

the show notes for this episode at

superdatascience.com/317. There, you will find a URL

to Edis's LinkedIn, any materials that we mentioned

on the show, plus the transcript for this episode.

Kirill Eremenko: As mentioned, make sure to connect with Edis follow

his career. I can already tell he's going to have a very

bright future if he keeps going the way he is going

now, and I'm personally very excited to see where it's

going to take him. If you want to meet Edis and lots

and lots of other inspiring and talented data scientists

in person, make sure to get your tickets to

Page 43: SDS PODCAST EPISODE 317: A DEEP DIVE INTO NEURAL NETS · career in data science. Thanks for being here today, and now let's make the complex simple. Kirill Eremenko: This episode

DataScienceGO, and we'll see you there. Until next

time, happy analyzing.