LOW-LEVEL C TUTORIAL - CS50 on Twitch, EP. 21 - VoiceTube: Learn English through videos!

Subtitles section Play video

COLTON OGDEN: All right.
Hello, world.
This is CS50 on Twitch.
My name is Colton Ogden, and I'm joined, once again, by--
NICK WONG: Nick Wong, hello.
That is my name, actually.
Sorry.
I guess I should start with that.
Oh, well.
COLTON OGDEN: What are we talking about today?
NICK WONG: So today we're going to talk about C, not C++, not C#, not Python.
COLTON OGDEN: A lot of C languages out there.
NICK WONG: C, yeah.
COLTON OGDEN: But C is like the progenitor-- progenitor?
NICK WONG: Yeah.
Progenitor, I think it was.
COLTON OGDEN: Progenitor.
These are the old fashioned ones, very simple, very lightweight
programming language.
NICK WONG: Yes.
COLTON OGDEN: And today--
NICK WONG: Fast is how it works.
COLTON OGDEN: --we're not going to-- yeah, very fast.
We're not going to be talking necessarily about the stuff
that we cover in CS50, though, right?
We're going to be doing maybe more of a deep dive?
NICK WONG: Yeah.
So we will open up GDB, which is the debugger for CS--
sorry, for C. Yes, for all of CS.
COLTON OGDEN: For all of CS.
NICK WONG: A granddaddy debugger.
We're going to walk through a little bit more in depth than CS50 does.
Although CS50 does talk about things like Malok
and memory spaces and things like that.
But I will also kind of be a little bit clearer on stack versus heap allocation
and why one was important, why the other one is not--
sorry, why any one is important at any one time.
We'll talk a little bit about buffer overflows and things
like that, as well as how input-output standard I/O works,
because I think that's really cool.
COLTON OGDEN: Yeah, yeah.
No, except low level, to a lot of people that are maybe not
super familiar with CS50 might think that means very basic.
NICK WONG: Easy, simple.
No, no.
That is a great misconception, I think, where a lot of programmers will say,
low level, high level, and then people are like,
oh, I want to learn high-level stuff.
And generally, that actually seems to be kind of true in the modern day and age.
A lot of people do like to learn high-level things.
But that is not because it's harder or cooler or anything.
High may kind of refer to, if you imagine a computer system
as at the lowest level, you have hardware, and at the highest level
you might have a graphic interface, and everything in between
is kind of organized.
Then that's what people are talking about.
So in the lowest level you have this literal, electrons running around
in, like, a piece of metal.
If you wanted to kind of vastly oversimplify and humorously
oversimplify what's going on, we basically took a rock,
tricked it into thinking, and then electrocuted it.
And that's the computer, right?
That is a computer's lowest level.
And then right above that, you have this kind of--
and we'll skip a few levels, just for the sake of time and clarity.
But there's basically machine code, and we'll kind of approximate that
as assembly, which is basically--
I'll show a couple of examples of what that looks like.
But the instructions, they're roughly one-for-one instructions
that a processor will execute.
And I say roughly, because that's not entirely true,
but it's the right concept.
And then maybe one step up from that, we might
have something like C, which is then going to be compiled down
into machine code or assembly.
And then that will get interpreted in some way.
And then maybe above that you might have something
like Python, which actually ends up kind of running on C. I mean,
it depends on the runtime you pick.
But it could actually be run through C.
COLTON OGDEN: Use C to compile the program that
executes the Python scripts, basically.
NICK WONG: Right.
Exactly.
Cool.
Awesome.
COLTON OGDEN: I'm very excited to dig into this stuff, because I
know C is a very popular language, or has been historically,
for game development.
NICK WONG: Right.
COLTON OGDEN: Assembly before it, for the old--
to tie back into, like, our NES stream.
NES was really in 6502 assembly--
NICK WONG: Oh, that's brutal.
COLTON OGDEN: --a very common older microprocessor,
still in use for small devices, but not as much
as I think more RMCPs these days.
NICK WONG: Yeah.
COLTON OGDEN: We have a lot of people in the chat.
I just want to shout out everybody who's joined us here in advance.
So scrolling way up, we have a lot of chat here.
So [? TwitchHelloWorld-- ?] that's [? JacksJPGuys, ?] [? ISOTV-- ?] let me
just keep scrolling here. [? Brenda, ?] [? RealCuriousKiwi, ?] was in the chat.
I see [? NonBored. ?] Thank you for joining--
[? Asley, ?] [? Nowanda353, ?] as always.
[? John09JLardinois, ?] thank you very much for joining today.
[? VippleBHJ ?] and [? Bhavic Knight-- ?]
[? Bhavic Knight's ?] a long-time regular as well.
Let me just make sure I haven't missed anybody else.
I think we had [? LKit57 ?] joined us.
I don't recognize that name.
I might just have a terrible memory.
But I hope I don't have a terrible memory.
Thank you very much for joining if this is your first time.
If not, I apologize. [? OoeyZuck, ?] Thank you very much for joining.
[? GoldenKappaChick-- ?] I'm not entirely sure what that means.
But I appreciate it.
Thank you very much.
Oh, we've got a lot of Kappas.
Maybe that's what-- maybe they're trying to do a Kappa test.
NICK WONG: Yeah.
COLTON OGDEN: Test and see if Kappas are working.
OK, [? JLardinois ?] was talking about [? AnnFogleman, ?] craft projects,
this is like the Minecraft C port, which sort of ties--
NICK WONG: I see, yeah.
COLTON OGDEN: --into this conversation.
He's saying he found a really good use for--
it looks like switch-case.
NICK WONG: Switch-case and break.
Nice.
COLTON OGDEN: Switch-case is a very, very common puzzle piece to use in C.
Everybody's hyped--
phones less delayed than PCs.
That's interesting-- higher the level, more abstractions.
NICK WONG: Right.
That is a great way of putting it, actually.
COLTON OGDEN: Yeah-- obviously trying to manipulate the Silicon directly with--
it's just you're going to be less productive than being able to write
a Python script, for example.
NICK WONG: Right.
Exactly.
COLTON OGDEN: So different layers of abstraction
allow you to do different things more or less easily.
MoHelat says, what can we do with the C language?
NICK WONG: So that's actually a really interesting question.
In talking about any--
they're called Turing-complete languages-- so basically,
any programming language you can think of--
you can do anything.
Anything that you can imagine being done with any language
can be done by any of the other ones.
So if I can do it in Python, I could also do it in C.
I could also do it using only NAND commands.
Technically, there is that possibility.
So generally, what ends up happening is, when people say what can we
do with language, or what do programmers do in a language,
they generally mean a little bit more like, what's convenient?
So what things have been written to be really, really good in that language?
What things have a bunch of APIs or libraries?
I guess, what has the language been really specialized for?
So when someone says that like R, for example, is built for biostatistics,
they mean, really, that there are a lot of developers who are in bio
and do a bunch of biostats who develop for R.
And those people are going to be a great community for that.
They have a bunch of resources.
They'll answer all your questions on it--
all sorts of things.
But that is not necessarily going to mean that you couldn't, in R,
build Minecraft, for example.
In concept, you could.
But whether or not you'd want to is--
I mean, I would argue you probably don't want to.
[LAUGHTER]
COLTON OGDEN: That'd be interesting to see.
NICK WONG: Right.
COLTON OGDEN: I'd be curious to see how somebody does that.
NICK WONG: --or like, a neural network in OCaml.
You can do it.
It would not necessarily be super fun.
Whereas in Python, It's pretty convenient.
And so these kind of choices-- they're based on convenience and comfort.
COLTON OGDEN: People have done a lot of hard work
making libraries and stuff like that, too.
NICK WONG: Right.
Exactly.
COLTON OGDEN: --people that have had an interest in solving these problems--
NICK WONG: Yes.
COLTON OGDEN: --put that hard work in in advance
to make it easy for everybody else.
NICK WONG: Exactly.
COLTON OGDEN: [? GravyWave123 ?] says buffer overflow.
NICK WONG: Yes.
We will talk about that, actually.
COLTON OGDEN: Why is C considered so fast once it is compiled?
Wouldn't any compiled language already be
compiled into binary, so therefore run with the same speed?
NICK WONG: Sure.
COLTON OGDEN: So that;s [? TwitchHelloWorld? ?]
NICK WONG: That is a great question.
So one of the things is that the developers of the C compiler--
if we're talking about CC or Clang, then that
has been just so thoroughly developed.
It's a brilliant compiler.
And we'll talk about compiler flags a little bit today.
And I think that--
there's a funny story dealing with compiler flags and the speller piece
in CS50.
But basically, the builders of that compiler-- the architects--
designed so many, just, brilliant techniques
for compiling from the C abstract machine down into assembly
that that ends up abstracting out-- or I guess not abstracting out,
but cleaning up a lot of your C code once you've
written it, to make it faster.
And one of the other problems that's faced by--
maybe you built another compiler and it's just as smart
as theirs, except maybe your code is like Python, where it's interpreted.
So every time I run it, it actually has to interpret the code that is being run
and then compile it as it goes.
So then that interpretation step actually
ends up taking up a bunch of time, as well.
And there's a couple other reasons that deal a little bit more with how C
integrates-- or C, C++, C#--
all integrate with--
I guess integrate's the wrong word.
It's not exactly what I'm looking for--
don't have as much overhead in what they do.
So for example, Java requires the entire JVM in order to run.
And that's a lot more overhead than C, which is literally just executing
instructions.
So there's all sorts of interesting things there.
There's a lot more complexity to that than I will be able to answer--
and probably--
guaranteedly more than I know.
[LAUGHS] But those are some reasons that come up to the top of my head.
COLTON OGDEN: Yeah.
So abstraction, in a lot of ways, you're paying for performance.
NICK WONG: Right.
COLTON OGDEN: Obviously electricity flowing through a rock--
NICK WONG: [LAUGHS]
More performance.
NICK WONG: Very fast.
You might not understand it, but it's very fast.
COLTON OGDEN: And to your point about NAND gates--
I think it was watching a video recently on Computerphile or something,
where they were talking about how transistors are purely built off
of just NAND gates, which I thought was fascinating-.
They wire them together in this weird way.
And that's all you need to do--
NICK WONG: You can do anything.
COLTON OGDEN: --to implement it all?
That's so fascinating.
NICK WONG: NAND on its own is Turing complete as a single instruction.
For those of you that are maybe not familiar with NAND, it is not of and.
And basically what that means is, if I put in a one and a one,
then I'm going to get the opposite of what and of one and one is,
which is one.
So I get the opposite of that.
That's zero.
And then if I put in a one and a 0, or a zero and a one, or a zero and a zero,
those are all zero when I do and of those three-- of those combinations.
And--
TOGETHER: That is going to be one.
NICK WONG: Yeah.
So NAND literally only has one case in which it returns zero.
The rest, it returns one.
And that allows you to do anything a modern computer could do--
COLTON OGDEN: That's outstanding.
NICK WONG: --which is crazy.
Although it can maybe result in millions and millions of lines of code.
Because if each line represents one NAND instruction, you have quite a bit
going on there.
[LAUGHS]
COLTON OGDEN: Yeah.
That would be interesting to see, actually, because I'm not
too deep into that side of CS.
But I would like to study more of it.
I thought that--
NICK WONG: It is very theoretical, yeah.
COLTON OGDEN: Do you ever watch Computerphile?
Have you watched their videos?
NICK WONG: I have not.
COLTON OGDEN: They have really fascinating videos.
NICK WONG: That's awesome.
Are they a YouTube channel?
COLTON OGDEN: They're a YouTube channel.
NICK WONG: Sweet.
COLTON OGDEN: Let me see. [? Brenda ?] was saying,
we have 35 new followers since yesterday.
Yes.
Thank you very much--
NICK WONG: Wow.
That's awesome.
COLTON OGDEN: --everybody who's following.
And I don't know if I told you, but we have over 1,000 as of two days ago.
NICK WONG: Wow.
That's awesome.
COLTON OGDEN: Followers--
NICK WONG: Thank you, guys.
COLTON OGDEN: So thanks, everybody, so much.
And also, thank you to the people who just followed.
So Visicks-- thank you for following.
And [? RasKWK. ?] "Ras-quick"?
NICK WONG: Ras-quick.
That makes sense, yeah.
COLTON OGDEN: [? FY6-- ?] these are some of the people
that followed a few hours ago, but just want to shout them out--
[? TaznorKoran13421 ?] and [? CodeBloodRexNanomizer, ?]
[? OoeyZuck, ?] [? FiveSharpPoint3, ?] and [? OldForce ?] and [? LOL. ?] Thank
you very much for following.
NICK WONG: That's awesome.
[CHUCKLING]
COLTON OGDEN: So let me just make sure that we're all caught up.
I think we're mostly caught up.
A lot of people are excited-- lots of cool stuff.
NICK WONG: Lots of cool questions.
COLTON OGDEN: Oh.
So A golden emote is a custom--
[? GoldenKappa ?] is a custom emote, I would say.
NICK WONG: That's sweet.
COLTON OGDEN: The real question is, can you build a neural network in Scratch?
NICK WONG: I think someone has.
I would not be surprised if there is a student from MIT who was like,
you know what?
I sat down and I built a neural network in Scratch.
COLTON OGDEN: Because they are like-- yeah, because MIT-- you got to rep MIT.
You got to do something crazy with Scratch.
NICK WONG: Yeah.
Yeah, exactly.
And MIT has this whole culture of building cool things just
for the sake of building them.
I actually really enjoy their--
I really admire their culture of doing that.
I think it's awesome.
We have a little bit of a culture here of that,
but I think we gear more towards the, can I get money out of it?
Nothing wrong with that.
But I think that I admire the kind of just
develop for the sake of development.
I think it's very cool.
COLTON OGDEN: [? MoHalas, ?] thank you very much for following.
And JP asked, is C a better language to learn in the beginning than C++?
NICK WONG: So to ask if a language is better to learn in the beginning
depends a lot about your context.
So if you have worked a lot with Python and Java,
then it might make a lot more sense to start the transition toward C++
because C++ has some of the tools and object-oriented perks of a language
like Java or Python.
But it is much lower level than either of those languages.
However, if you are starting from nothing,
CS50 seems to think that C is a great place to start.
And I agree.
And we'll actually talk a little bit, when
we get into coding some stuff, why I think
it's a great language to start with.
In particular, I think it helps you think a lot more like a computer.
And that tends to be one of the problems with becoming a computer scientist
or starting in CS, is, computers have access to a lot of tools
that we don't as human beings.
And they think-- er--
think-- I guess until the singularity, I'm going to not try and personify them
too much.
But they kind of process things in a way that is very different from ourselves.
And it's important, as a computer scientist, to realize that.
And even myself, having studied for a couple of years,
I still find myself realizing new ways in which computers process things
differently from me.
And that's very cool.
So I would argue that C is a great place to start.
C++ also would work.
It does a lot of the same things.
But it depends where you're coming from.
COLTON OGDEN: And [? JPGuy ?] says, thanks for clearing that up.
Let me just make sure.
I don't want us to spend too much time before I actually start diving in.
People are saying they recognize Computerphile.
Oh, and then [? UnsignedEd ?] wants us to go into a bitwise operators.
NICK WONG: Excellent.
I'm really glad you asked about that, because I
used to think that was one of the most confusing things
that I thought people just wrote just to be confusing.
When I started in CS, I would look at code
and think that it was written to be confusing for the sake of being
confusing.
However, that is not true.
There are often a lot of reasons for why people make design choices.
And so we will talk about, like, bitwise,
and why it might be really fast or why it might be really useful.
It will deal a lot with how we go into assembly.
COLTON OGDEN: Cool.
And then [? DyGween ?] says, hey, guys.
I'm a CS student from Rio de Janeiro, Brazil.
NICK WONG: Oh.
Sweet.
COLTON OGDEN: So thank you very much for joining-- appreciate that.
And it looks like they're talking in Portuguese as well.
And [? DroveForce ?] says hello from Mumbai, India.
NICK WONG: Wow.
That's awesome.
COLTON OGDEN: Worldwide representation and [? BellaKeers ?] is joining us.
Hello, [? BellaKeers, ?] thank you very much.
Why don't we dive into some of the topics I want to take a look at?
So I'm going to switch over to our handy-dandy computer view--
NICK WONG: Favorite screen saver.
[CHUCKLING]
I think as a tradition now, I'll just open up with this every time.
I mean, I'm actually going to be working--
thank you.
Or I guess the developers of that should say thank you.
I'm going to be working in a virtual machine because--
COLTON OGDEN: I was just about to say, I don't recog-- this is Mac OS.
NICK WONG: Yeah.
This is not Mac OS.
This is Ubuntu, the desktop version.
And I'm going to work in a virtual machine
because I like separating out my C development from my actual computer.
And also, it's a little bit more convenient.
There's a lot more guides on how to download
C stuff onto a virtual machine.
I also happen to use this virtual machine for my systems class,
so I have and do have a lot of C tools on it already.
COLTON OGDEN: Nice.
NICK WONG: And it's super convenient.
So that's what we are currently on, if you want to follow along.
COLTON OGDEN: There's a very large, lovely prompt you have--
NICK WONG: Yes.
We have quite the prompt going on.
I was kind of playing around with it right before the stream
because I realized that my prompt was enormous.
It was so long because I like to have a lot of information on my prompt.
But it's not super useful to you guys.
So I killed all that.
Now, you just have the current directory and an equal sign-- greater than sign--
as my prompt.
COLTON OGDEN: And [? JPGuy's ?] saying, that font's nice and big.
NICK WONG: Yes.
You guys should hopefully be able to read that.
If it's too big, and we find that it's making things inconvenient,
I can always shrink it.
But for now, hopefully this works.
[CHUCKLING]
All right.
So in C-- we'll talk about make files in a little bit,
but otherwise, we're actually going to start
with just raw compilation using CC.
And we're going to do all sorts of cool things.
But the first thing we're going to do is write our first C file.
So I'm going to use Nano as my text editor of choice.
And we're going to say, you know what?
Let's make our--
I don't know.
We'll call it first dot C.
COLTON OGDEN: Do you use Nano a lot for actual development,
or are you more of a Vim user?
NICK WONG: Not usually.
I tend to use, actually, Visual Studio Code for actual dev.
But if I'm typing as I go, then I tend to use Nano very quickly.
COLTON OGDEN: And then, do you use VSCode in your Ubuntu?
NICK WONG: I do, actually.
So I have VSC.
I can do code dot, and that will open that up for us.
And that helps me when I'm doing any sort of integrated development.
And we might actually switch over to that if we need to-- yeah.
Actually, this might be a good idea.
We're going to switch over to VSC, so I can keep code open and what's going on.
So here we go.
We have a new prompt out here.
That might be a little bit too small.
We'll open things up a little bit.
Where is-- should be under View.
See, the problem is--
COLTON OGDEN: That's what I would imagine, but yeah.
NICK WONG: Yeah.
I don't see it.
COLTON OGDEN: Everyone's going to have their own settings, locations,
and whatnot.
NICK WONG: I know.
Everybody--
COLTON OGDEN: [? And JPGuy's ?] saying no, use Visual Studio Code,
[? Kappa. ?]
NICK WONG: This is Visual Studio Code.
Oh, you're talking to [INAUDIBLE].
COLTON OGDEN: In Vim, we have [? Vimer ?] now.
NICK WONG: Yes.
COLTON OGDEN: [? Vimer ?] kids, if you're scared of Google,
then your best bet is Microsoft.
NICK WONG: Yeah, OK, I could agree with that.
COLTON OGDEN: [? NonBored's ?] saying they're a vi guy.
No!
Use Vim says JPGuy.
NICK WONG: I don't use Vim.
So Vim and vi are quite fast.
And if you want to be well respected, I guess--
not well respected--
COLTON OGDEN: [LAUGHS]
NICK WONG: --but a lot of coders will prioritize those.
COLTON OGDEN: Some of them do get brownie points for Vim users.
This is definitely true.
NICK WONG: Which-- yeah.
Honestly it's deserved.
COLTON OGDEN: Whether it's meritorious or not is up to debate.
I think the difference-- as long as you're using a tool and doing work with
it--
NICK WONG: True.
As long as you're building things--
COLTON OGDEN: Everybody has preferences.
NICK WONG: It's true.
COLTON OGDEN: I'm a VSCode user myself.
NICK WONG: Yeah.
It's great.
Yeah.
We actually ended--
COLTON OGDEN: Were you using VSCode before the games course?
NICK WONG: Yes.
I actually was.
COLTON OGDEN: How long have you been using VSCode?
NICK WONG: I started using it, I think, right after someone showed me Sublime.
Because I was--
[LAUGHTER]
I switched off of the CS50 ID.
And someone was like, oh.
You should use Sublime.
And then it kept asking me for a subscription.
And I was like, well, this is dumb.
And I looked up for--
COLTON OGDEN: It's kind of irritating.
NICK WONG: --free IDE.
That's great.
And it was like, Atom and VSC came up really high.
Someone suggested Emacs, which is also super powerful.
A lot of my professors use it.
But I do not.
COLTON OGDEN: People in the closure community are in love with Emacs.
NICK WONG: Yeah.
I think it integrates really well with a lot of things.
COLTON OGDEN: It was written in Lisp, which is--
NICK WONG: Which would make sense.
[CHUCKLING]
COLTON OGDEN: Closure people probably like it a lot.
Anyway--
NICK WONG: Yeah.
That would explain a lot.
[CHUCKLING]
COLTON OGDEN: We're getting off on a tangent.
I apologize.
NICK WONG: No.
It's all good.
I think that's one of the best parts of these streams.
So we have our first C program, and it ends in the dot C extension,
although, technically, you could end it whatever extension you want.
And we're going to do is the characteristic int main void--
oh, man.
Boom.
And this is technically a C program.
This, actually, will compile just fine.
Keyboard shortcuts are a pain.
So technically, I can compile this into first, and I'll use first dot C.
And that compiles just fine.
I can even run it.
That's totally reasonable.
It does absolutely nothing--
COLTON OGDEN: This will still be sort of covered in CS50.
It's just going to be using a compiler--
NICK WONG: Basic C.
COLTON OGDEN: --C code into object code-- dash o-- yeah.
NICK WONG: Oh, right.
And we do actually cover quite a bit in CS50.
I don't even know if we're going to touch too much on objects.
We might touch on, libraries and things.
COLTON OGDEN: Oh.
That's true.
NICK WONG: I'll focus a little bit more on data representation and things
like that.
But yeah.
So this is technically a C program.
It does literally nothing.
But-- just kind of as a reminder to everyone-- this int main void--
well, int main is a very important part of C programs, in the sense
that you need it in order to run.
That's going to be the main thread, for example.
So I'm going to import some libraries.
Losing my mind-- for example, standard IO
is, I think, a fantastic library to use.
And we're going to talk about it quite a bit.
And what we can do is, we can also accept command line arguments.
So we're going to allow for that.
And we'll do [INAUDIBLE].
Whoops.
I hate doing that.
That's a command shortcut from Mac, but I always
forget that it doesn't work on this.
And now we have access to all of these things.
If there is like a variable that you pass in,
and you don't want to really use it yet, and you want to start for later,
casting to void is a non-trivial operation
but it allows us to ignore this unused variable warning
that I will point out in a second.
Oh.
Well, OK.
There should be a warning if I was using compiler warnings.
I think dash w--
unused-- I can't spell--
unused variable.
That might not be correct.
Whoops.
Oh, well.
You can pass all sorts of flags to the compiler.
There's hundreds.
I don't know all of them off the top of my head.
And I don't really use a lot of them off the top of my head.
I just use whatever standards are set by the class I'm in.
But they are good to know.
And we're not going to really play around too much with them.
But if you have unused variable warnings,
you can always cast avoid to ignore them.
That's generally a development practice rather than a production one,
but something that is, I guess, good to be aware of.
So this program that we're going to run at the beginning is--
let's say that we want to--
I guess we're just going to hop right into it.
I want to be able to filter Python files.
I'm really annoyed at my developers for writing thousands and thousands
of lines of comments.
And I want to filter out all of their comments.
So what I'm going to do is, I'm going to allow us to open up a Python file.
And then I'm going to just filter out those lines.
And shouldn't be too bad.
All I have to really do is include our standard-- actually,
I think standard IO should suffice.
So what I'm going to do is--
we'll just take that first argument from argv.
And we will open it as a file descriptor.
So file star-- in file--
is equivalent to fopen of argv1.
And we are going to add in fclose--
actually, I think it's just--
oh. fclose works-- infile at the end.
And what this allows us to do is read from that file.
So all I'm going to do is read out the first, we'll say, 8 bytes of that file.
So fprintf to standard out-- oops--
of the first - bytes of that file.
And this is a reasonable way to do it.
And you might go, what's buff?
And I will then tell you, yes.
Of course.
Great question.
Buff is an eight-byte-long array of chars.
And what I'm going to do is throw in--
COLTON OGDEN: Buff's short for buffer, using it as a buffer.
NICK WONG: Yes.
We're going to use it as a buffer to store whatever we want to deal with.
And then I'm going to open--
let's see.
We already opened the file.
And then what I can do is read in--
man, I can't remember the name of this.
Ah, read, right.
[CHUCKLING] Correct.
And read, I believe, is source destination.
Let me just check that.
COLTON OGDEN: It's curious about functions in the C programming
language.
NICK WONG: Yes.
COLTON OGDEN: [? Man ?] were [? the Man ?] I guess, program.
[INAUDIBLE] looked them up.
NICK WONG: Yes. [? Man ?] is a fantastic program.
It allows you to do quick checks like that
if you forget which order things are as far as arguments go.
And this should be totally fine.
COLTON OGDEN: Famous last words.
NICK WONG: Yeah.
That's never true.
But in concept, it's the right idea.
So we've now compiled first.
And now, if I run first, this will break--
well, I guess not break, but we're not going to really do anything.
And that's strange.
Because I didn't pass in a file, but I'm also not checking for things.
And if you took CS50 or if you were following along,
then you would know that we actually usually
required that you check to make sure that this sort of thing doesn't happen.
Accessing argv 1 without actually necessarily having one
can be kind of weird, and undefined, and freaky.
So we're going to do is--
if argc is not equal to 2--
one for the file name and one for the actual name of the command being run--
then we're going to fprintf to standard error.
And we're going to say usage is %s filemame.
New line.
COLTON OGDEN: And for people unfamiliar, argc is the number of arguments
that you passed--
NICK WONG: Correct.
COLTON OGDEN: --at your program that you wrote at the prompt
when you executed your program.
And argv is an array of strings that represent those individual arguments.
[? OPLAD11874, ?] thank you so much for following.
NICK WONG: Where is that file?
I think it's actually just from file IO.
[INTERPOSING VOICES]
That's the other thing.
It's the read function.
I always forget where those all come from.
I feel like file dot h is maybe not right, but maybe it is.
Who knows?
We'll find out in a second.
No, that's wrong.
COLTON OGDEN: Well, because, didn't it compile when you had it written.
NICK WONG: It did, but it was just being dumb.
It's because I'm not passing a lot of the same--
with unused variable-- with specific warnings-- flags to the compiler.
And so that's where we're going to use Google--
where I don't want to include POSIX.
I want to include--
COLTON OGDEN: Oh, no.
We're back in Mac OS X.
NICK WONG: File operators.
Yeah.
This is Mac OS--
C library.
A lot of times, these sorts of things, I just forget what they are.
Oh-- standard IO, I think, is--
oh.
I thought that's what that was.
I thought read was part of standard IO.
In fact, I'm fairly certain it is.
COLTON OGDEN: And this is a common lesson, too.
You don't need to memorize all of the functions--
NICK WONG: Right.
Exactly.
COLTON OGDEN: --that exist in C. The important thing is that you understand
the syntax and how to solve problems.
And then you can always--
modern developers have the luxury of just being able to Google--
and also, modern developers are building larger, arguably more complicated
systems than they were in the 80s and 90s.
Maybe not--
NICK WONG: Right.
Exactly.
COLTON OGDEN: It depends on how you define that.
But I think-- definitely using a lot more libraries and having to memorize
a lot more functions--
NICK WONG: Right.
COLTON OGDEN: --I think is probably, definitely the case.
So give and take.
We have Google, but we also have millions of functions and libraries
to sort through.
And it's impossible to remember all of them.
NICK WONG: Right.
COLTON OGDEN: Like never--
NICK WONG: I mean--
COLTON OGDEN: Even the ones you use on a daily basis,
like, what arguments go into this function in what order?
Doesn't matter as long as you know what the function does
and how to use it to solve a problem, right?
NICK WONG: Right.
Exactly.
So let's see if we can figure out what's going on here.
Actually, I can switch to fread and have that be from standard library.
That would be fine.
COLTON OGDEN: That's what [? RasKWK ?] just said, too.
NICK WONG: Oh.
Nice.
COLTON OGDEN: Fread from [INAUDIBLE].
NICK WONG: We agree, which is good.
COLTON OGDEN: It's like ESP.
NICK WONG: Oh-- or not.
What does fread require?
This is where things are kind of fun.
I can use that from standard IO.
Oh.
I see.
So what they have you do--
sure.
OK.
That's fine.
So we'll use fread from standard IO.
And what we will do is have that the reading into the buffer from here--
eight bytes of size--
or, sorry-- size 1--
8 bytes.
And we'll have that read from in file.
And that should be fine for us.
Fopen is also too few arguments.
Right.
That is a great point and I totally neglected it.
We're opening things up for reading.
I don't remember if it takes a standard.
COLTON OGDEN: I see you're used to using single quotes.
NICK WONG: Yes.
[CHUCKLING]
COLTON OGDEN: Python or Javascript developer.
NICK WONG: Single characters-- and I think it requires a constant char star.
So that would be really bad.
And now we should be OK on each of those things.
And we get a segfault. And so basically, what's going on here
is, you might go, well, I shouldn't have gotten a segfault. I caught the error.
And then you'll see in the same place where this is what's going on--
where we actually should have exited with some error code and then moved on,
right?
I keep forgetting which language I'm writing in
as well as where I'm writing things in.
Oh, and is from standard library.
There we go.
Now we're all on the same page.
A lot of these things I don't necessarily
memorize for the same reasons that Colton pointed out.
If we were writing in a language that I need to write in consistently for work,
then I would naturally memorize most of these things.
But otherwise, I'm not going to generally.
COLTON OGDEN: There's only so much finite short-term memory.
NICK WONG: Right.
Exactly.
And so if we were writing in Python, no problem.
I can deal with that, basically, at any time.
But if we are actually writing in C, then I
think it's just fun to play around and see what's going on.
So we now have a functional program, we think.
But if we were going to go through here, it
would be a decent idea to check that fopen didn't fail.
And generally, whenever you're doing these syscalls and things that
are going on, then I would actually be very careful about making sure
that a syscall didn't fail.
So for example, if fopen fails, we might go into man fopen
and see what ends up happening.
Sorry.
My screen is a little large.
But if you go down to the return--
man pages have a pretty standard format, so you can actually
usually guess where things go.
And we get that return value.
So upon successful completion, it returns a file pointer--
yeah.
Not a file descriptor but a file pointer.
Otherwise, null is returned, and errno is set to indicate the error.
So what we can do is, then, we can say, if not in file then
I want to fprintf to standard error--
some sort of error number.
Or we'll say, error blobdeblah, and file not found maybe.
And then there's actually something really cool you can do there.
But we will talk about that in a second.
Errno, and then we're going to exit.
And this might be with a separate error--
or I'm going to actually just pass errno.
Errno should be set, but we'll find out in a second.
Oops.
Compile that-- of course, where is errno to click?
There are so many ways in which I don't like that.
We'll deal with this later, then--
file could not be opened.
And we'll just set error to 2.
And then what we can do here is compile it.
And now, if I tried to open a file that doesn't exist,
the file couldn't be opened, which is good.
So let's see-- I read in the chat, [? TheDuckThough ?] says,
don't do exit, kids.
It will not run any deallocation and leak a lot of memory.
Well, that would be a good point.
I don't know of any, necessarily, accurate functions
that do automatic deallocation and non-leaking of memory.
If you're working in a low-level language,
you should make sure to do your own deallocations
and non-leakages of memory.
In this case, it would be impossible for me
to have a leakage of memory in the sense that I never allocated any--
at least, I didn't manually allocate anything to the heap.
So it's not a problem here.
There is-- if you notice at the very beginning,
I went with underscore exit, which doesn't do any of its own cleanup.
And the reasoning for that basically being
that generally, when I'm coding in a low-level language,
I'm expected to do my own cleanup manually because for courses, they
don't want you just using library functions that
can help clean everything up as you go.
So that is what the purpose of that function is.
And so it's generally a good idea to be very aware of what exactly
is going on when you call any one of these functions.
So we're going to talk about that a little bit.
We're going to use strace as well to talk about where
actual system calls are going.
But this start up process is always a pain for me, regardless
of when I actually first start writing.
And I don't usually write C files completely from scratch like this.
I generally actually pull template code that I just store somewhere.
So I'm not actually used to this format for doing things.
We're actually going to include errno.
I forgot that it is a library.
And there's a really cool function that I can't remember the name of.
And let me see if I can find it because--
and this might seem a little bit ridiculous-- but the reasoning for that
is that it allows you to describe an error using a string.
And I don't remember what that function is called.
So we're going to Google that because welcome to development.
Errno-- string description.
COLTON OGDEN: [? C9221, ?] thank you for following.
NICK WONG: Strerror-- that might be what I was looking for.
Well.
OK.
That might be true.
We'll try that.
So this is part of C++, as far as this goes, but it might be part of it.
We'll find out.
Let's go see.
So these are all kind of like non-, I guess, important--
what was that called?
Strerror.
I feel like that exists.
Nope, that's not a library.
That's probably an errno.
Hopefully it is.
If not, I won't spend too much time on it.
But it is kind of a cool thing that you have access to and don't
have to really deal with on your own.
Ah.
There we go.
Sure.
Oh, no.
That's not what I want.
You are-- Does it type int?
Really?
All right.
Well, then, we're not going to spend too much time on it.
There is a function that exists like that,
and I apologize for not knowing off the top of my head.
It's there somewhere.
That's OK.
So-- oh.
That was not what I wanted.
There we go.
So now I run this.
It tells me which error I got.
And then we're all good from there.
COLTON OGDEN: Error 2--
NICK WONG: Yes.
Error 2, which is what it was set to anyway, but whatever.
So now, ignoring that part, I'm going to answer some of the questions
that I see upstairs--
or not upstairs.
COLTON OGDEN: Upstairs?
NICK WONG: In front of me--
COLTON OGDEN: I've gone back--
NICK WONG: There's quite a few questions.
COLTON OGDEN: So [? ThePeeingHorse, ?] which is an excellent name--
[LAUGHTER]
--says, people learn the C language when there's GO?
That blows my mind.
What are your thoughts on that?
NICK WONG: So it's interesting, because in the C language,
you are actually kind of stuck to this whole--
you get a specific stock allocation.
You can't modify that allocation.
You can't take some thread stack and move it to somewhere else.
In C, you are stuck with whatever stack you're
given, because the actual stability of pointers matters a lot.
However, in GO, that's not true.
You can move things around.
It's a garbage-collected language, and there's a bunch of stuff
that I don't understand about how that works.
And you can move that sort of thing around however you'd like.
And that's really cool, but I think that starting with C is extremely useful.
And technically speaking, if you're talking about which language
you want to pick, people will always have
their own opinions on where you should start and things like that.
My opinion is that starting with something like C is totally reasonable.
It teaches you how to think like a computer.
It puts you in charge of all of your own memory allocation and it allows you
very low-level access to a computer--
which, I think, if you accomplish all those things,
it's a widely-used language.
There's well-developed support for it.
And there's a community that knows exactly what they're
talking about that has been around for, literally, decades.
And so I think that all of those attributes
make it a really desirable language for learning at the beginning.
I also would argue that learning a specific language doesn't necessarily
matter, in the sense that, as long as you're starting to think like
a computer, and as long as you're starting to actually think of things
algorithmically, and efficiency-wise, and as thoroughly as you possibly can,
then you are developing the mind of a thought process--
not of a thought process--
of a computer scientist--
COLTON OGDEN: That's kind of meta.
NICK WONG: --and that thought process there-- yeah.
You're developing the thought process of a thought process--
You're developing the mind of a computer scientist.
And I think that that's what's important.
COLTON OGDEN: Yeah.
And there's also no reason that they're mutually exclusive.
You can learn GO and C. And maybe even--
NICK WONG: Right.
Right.
Just use them at the same time.
COLTON OGDEN: And I'm sure that GO was actually implemented on top of C,
or at least, well--
NICK WONG: I don't know.
Yeah.
I don't actually know--
COLTON OGDEN: It could be.
I mean--
NICK WONG: It might have its own compiler.
Yeah.
COLTON OGDEN: It could have its own compiler.
But yeah.
I think that religious arguments with languages I don't think
is super productive.
People can learn multiple--
NICK WONG: Can learn all sorts of things.
[CHUCKLING]
COLTON OGDEN: It's good to learn multiple programming
languages-- different ways of thinking.
NICK WONG: Yeah.
Exactly.
COLTON OGDEN: People had some cybersecurity comments as well.
Brenda was joking, you don't follow the CS50 style guide.
NICK WONG: Right.
I believe the CS50 style guide actually does something that I hate--
very strongly dislike-- which is, they do this--
which I detest.
However, as a TF-- as someone grading things and reading through code--
I love that because it is so much easier to read.
However, I used to love it.
I now don't really, particularly care, and I think this looks cleaner.
I think that having this on its own line is just kind of strange to me.
So I generally prefer this style.
COLTON OGDEN: This is the original Kernighan and Ritchie--
NICK WONG: Right.
[CHUCKLING]
COLTON OGDEN: --style, mostly because, at the time,
they only could fit like 10 lines of code onto the screen.
But a language that I've been diving into recently is C#.
And in the C# community, you are expected to do the--
NICK WONG: This one?
COLTON OGDEN: Yeah.
And if you're writing in people's code, there
are certain standards and expectations in the community.
NICK WONG: Right.
Right.
COLTON OGDEN: And if you don't follow those, you're kind of an outlier.
NICK WONG: You get kind of pushed in your [INAUDIBLE]..
COLTON OGDEN: It's the same thing in Python
how I used to get scolded for using Caml case notation for variable names.
NICK WONG: Right.
Yeah.
No.
That's terrible.
But if you're coming from Swift or Objective C--
COLTON OGDEN: Yeah.
People were very upset about that.
Also shout out to--
David's in the chat. [? DavidJ.Newlon. ?] Tabs
or spaces, Nick, says David.
NICK WONG: Tabs or spaces?
I would thoroughly argue, spaces--
portability and making sure the Python stops yelling at me.
COLTON OGDEN: Yeah I mean, modern text editors just let you say implement--
NICK WONG: Yeah.
Actually, I was going to point out--
if you look down in the bottom right where I am currently sitting--
we'll see if I can hop out of there--
where my mouse is and where I was--
spaces is set to 4.
So all of my tabs actually are spaces.
I hit tab out of convenience, but generally I try to be very consistent
and use spaces only.
COLTON OGDEN: Yeah.
I would hate having to type four spaces every time, every line--
or eight spaces or 12 spaces--
NICK WONG: Right.
Exactly.
It's a huge pain.
COLTON OGDEN: Tabs that are implemented as spaces are definitely the way to go.
NICK WONG: I'm a huge fan of that.
COLTON OGDEN: Let me make sure we didn't miss any other key comments up above.
[? JPGuy's ?] saying they don't develop in C,
even though they been a lot of time here.
I mean, we don't really spend a lot of time doing C on the stream.
I mean, obviously, C--
NICK WONG: No, no.
COLTON OGDEN: [? It's 50/50 ?] C, but as kind of a learning point,
and they've sort of let you jump off into Python, JavaScript,
other languages that are actually more in vogue, I think,
for most developers these days.
Aqualad11874, thanks for streaming.
Thanks so much, [? Aqualad, ?] for joining us,
and thank you for the follow earlier.
People were helping you debug live.
[? UniSTD.H-- ?]
NICK WONG: I always appreciate it.
COLTON OGDEN: [? --was Brenda ?] said it.
I'm not sure what that might have been a reference to.
NICK WONG: There's all sorts of cool libraries for things.
I tend to stick to very simple, very minimal libraries.
Well, I say minimal, standard libraries is huge.
What I basically mean by that is I stick to things
that are as simple as I can find them, or as close or low level
as I can find them.
And generally, the reason for that is that I
wants to be very consistent in how I actually end up
developing an any piece of C code.
I also try not to include libraries if I don't have to.
And people would argue that there's like, efficiency
purposes to that and things like that.
That's true, you know, with a caveat.
But it's more important to me as a coder that I don't include a bunch of fluff
everywhere that people don't need to read through,
or that makes it harder for you to process where things came from.
And for debugging purposes, if I include, like, 10 different libraries,
I might not necessarily know where any one function came from,
and it's easier for me to just keep in my mind what's going on.
And especially since you guys are getting
kind of a live version of how coding works here,
you basically are seeing a little bit of how my development process works.
Which is very-- tends to be very--
COLTON OGDEN: Very Google-oriented.
NICK WONG: It's also being very Google-oriented.
And I saw someone asked--
what was the question-- oh.
Someone asked which programming language--
or did I learn a single programming language for technical interviews?
And the answer is I actually learned programming languages because I
thought building things was cool.
My focus at the very beginning, I don't even
know what a technical interview was.
I had no clue.
I just knew you somehow could end up working at big tech companies
or whatever that was.
But I just thought was really cool, I like building tools for varying things.
And so I kind of accidentally learned a lot of languages.
I learned a lot of them through my classes, I learned a lot of them
just by like wanting to build x and knowing
that I needed to hop through learning this language to get there.
But the language itself was usually not my goal.
I actually don't have a single language yet that I've learned
for the sake of learning that language.
However, I think this winter break that might change.
COLTON OGDEN: Which language are you--
NICK WONG: I think I might actually learn
Go, for the specific reason of thinking that it's a very cool language.
COLTON OGDEN: It's apt because you going to work at Google, so I mean--
NICK WONG: Right, I think that would make sense.
And so I think that that is--
that's probably something that'll change a little bit.
But in general, that's still going to be very project-oriented.
I think that it's a little bit easier to motivate yourself
if you have project-oriented things going on in your mind.
And also, something that's interesting about technical interviews
is a lot of companies that I've interview at,
I haven't really needed to code in a specific language.
With startups that's a little different.
But with big tech companies--
COLTON OGDEN: Startups have more of a specific stack
that they're looking to--
NICK WONG: Right, exactly, they're like, I want people who can do this,
because that's what our team is currently working on.
COLTON OGDEN: It's fairly small, it's fairly focused.
It's not as wide-reaching as like--
Google obviously has a million different things.
NICK WONG: Right, you can go develop with.
Anything and so I think generally speaking, I will learn kind of--
I mean, I learn languages because you do need them to write actual code.
You can't write in pseudocode.
Well, Python, basically.
But you can't otherwise do that.
COLTON OGDEN: Yeah.
NICK WONG: But I love Python, I use it all the time.
COLTON OGDEN: Python's a great one.
NICK WONG: It's great.
COLTON OGDEN: I love Python.
NICK WONG: But I generally, in a lot of technical interviews,
I kind of write in a blend of languages.
There's certain things that are really convenient in C,
so I might talk about them, and there's certain things
that are really convenient in Python, so I might kind of like
ping something towards Python.
The funny thing is like, if you look at my like whiteboard interviews,
I'm pretty sure I like, every other line,
have a semicolon at the end of the line, and like--
[INTERPOSING VOICES]
Yeah, it's like, I don't know where that language came from.
That's from C, C++, and Java.
And so it's very interesting.
Oh, someone also-- while we're on the topic of technical interviews--
asked, can you look at these libraries during your technical coding
interviews?
No, not usually, at least not in in-person interviews.
And I would also recommend not really doing that.
You don't really need to, because I think
that you can-- an interview I had, I said, well, you know,
there's a library that does this.
I know it exists.
I don't know the exact syntax, but I'm going to write it like I do,
and you'll understand what I mean.
I'll use clear syntax and clear things in order
to indicate to you, my interviewer, what I'm trying to do,
because the interesting part is solving this problem.
That's what I'm after, that's what you're after, and you know,
you have four interviews today.
Hopefully, this will be interesting.
Because I want to solve the problem.
I don't need to focus on, is it a library that's capitalized or not?
Who cares?
I can go Google that.
And so generally what I would do there is I'll say,
if you don't-- and it's called a library.
You know, let's say I forgot Malok, which that might be kind of a red flag,
but like, we'll pretend that's not.
And I say, we're just going to call it the Allocator library, and I'll say,
here's the syntax for it and concept.
It has some sort of allocate, it as some sort of reallocate,
and it has some sort of delete.
And so that's kind of a weird mixture as far as syntax goes,
like a purist would say, well you're blending C, C++,
and like some weird version of Python.
I mean, that's fine.
I'm still hitting the concepts.
I'm showing and demonstrating that I understand, what does it
mean to allocate it on the heap?
Things like that, I think, are really useful.
I think those concepts are much more useful than actually
memorizing all of those things.
[? TwitchHelloWorld ?] says that that's interesting.
I have to be super exact and technical in interviews in my area, though.
Only for teaching interviews do I get asked
to actually do something similar to a coding interview.
Yeah, my pleasure.
Oh, they said thanks, and I said yes.
COLTON OGDEN: [? HongYi ?] and [? ImagineAWave, ?] thank you very much
for following us.
NICK WONG: Yeah, awesome.
COLTON OGDEN: Let me just see.
We have a lot of people--
[? AraniAndAndre, ?] I noticed during the chat.
[? TheDuckThough, ?] I think we saw [? TheDuchThough ?] earlier.
Do you still use the auto and register keywords for C50?
I'm not sure if--
oh, well, that's for C++, newer C++.
NICK WONG: Oh, yes.
COLTON OGDEN: [? FCtrl.H. ?] [? CRonics ?] made a point.
I'm not entirely sure what library that is.
NICK WONG: Yeah.
COLTON OGDEN: "Using libraries is for the weak."
NICK WONG: And I think we kind of hit on something similar in the last,
maybe two streams ago, where it was like.
If I built everything myself, would I be secure?
And I'd argue, no.
And so I mean, that's kind of interesting.
But yes, using libraries is for the weak.
I count myself amongst the weak.
I also sleep, which is for the weak.
COLTON OGDEN: Why exit instead of return?
NICK WONG: Right.
Basically, that just comes from a old habit of mine,
where I am actually in the process of completing a bunch
psets where we were dealing with process forking and threading.
And so I was using exit from those psets,
and I just kind of got into that habit.
There's not necessarily a huge distinction in this particular case,
because we have a single thread, we have only the main thread,
and exiting from it as well as returning from it
achieved roughly the same goal in that they both end the function.
A return, it's going to return whatever that number integer you have out to,
I guess, the kind of error return here.
This.
And it'll deal with a lot of just kind of what process cleanup has.
I like exit.
It does do some of its own cleanup.
It deals with a little bit of marking a process as dead
and getting it killed and things like that.
But in this case, I don't think it actually really matters too much,
and for our purposes it certainly does not matter.
COLTON OGDEN: All right, cool, cool.
Let me just make sure-- we have, oh, there is so much chat.
NICK WONG: Yeah, we really appreciate you guys for participating.
COLTON OGDEN: It's amazing.
We're super happy.
I feel horrible that we haven't been able to read off
every message at this point.
And [? Eddie was ?] saying, C++ isn't [INAUDIBLE] either, which is correct.
[? Fatma, ?] hello, thank you for joining.
"Hey, [INAUDIBLE],, what are the benefits of using
float instead of double, which calls for higher precision.
Just the memory usage?"
Asks [? PresidentofMars. ?]
NICK WONG: So there can be a bunch of reasons
for using a float instead of a double, or vise versa.
Generally speaking, I will argue for using
as little memory as possible and as simple as possible.
So if you're talking about some sort of float,
then we're dealing with a fairly large amount of memory
relative to an int or a char.
If you're talking about a double, then you're similarly in a boat
where you're using quite a bit of memory relative to an int or a char.
And I actually generally don't really use float or double--
I mean, that's a weird statement.
COLTON OGDEN: [INAUDIBLE].
NICK WONG: Yeah.
It's a weird thing to say.
COLTON OGDEN: Integers all the way.
NICK WONG: Just all integers.
And I think that there are, I guess, just use
cases in which either one is useful.
Sometimes it deals a little bit more with which platform you're on, but even
then, in the modern day and age, that's not really true.
I can't think of any use case right now.
If I think of one when I get there, I will let you know.
I think it's a good question that broadens to kind of,
why do any one practice over others?
So for example, why would I stick to using integers and chars?
Integers don't cover enough numbers, and chars are only one byte.
But, for example, we're going to talk about how one of the powers of C
and low-level languages in general, is I get direct access to memory,
and I can think of it as kind of like a raw block.
It doesn't have any shape, it doesn't have anything
that it really does on its own, and I can reshape that block however
I'd like, which can be really useful.
So that can be really interesting to do, and we'll get there
in probably a couple of seconds.
Well, a couple minutes.
COLTON OGDEN: Thank you very much, [? DwayneYasir ?] for joining us.
I know that--
I don't know which environment I saw it in recently,
but depending on whether you have a 64 or 34-bit processor,
doubles are actually more efficient than floats,
because they're typically 64 bits.
NICK WONG: OK, I can see that.
COLTON OGDEN: So they're the--
[INTERPOSING VOICES]
NICK WONG: OK, I can see that.
COLTON OGDEN: So it can be loaded in one CPU call.
And I forget whether it was in--
I think it applies both to the content of C, C++, and also the VM platforms,
like the CLR and Java, JVM.
They're able to just more-- especially if you're
doing a lot of double operations in a tight loop.
NICK WONG: Right.
COLTON OGDEN: Loading that double into--
NICK WONG: Can be very annoying, yeah.
COLTON OGDEN: That's the only context that I'm aware of, besides the fact
that you get more precision in with a double, where you can actually
get performance costs out of it.
But to your point, I don't think for a lot of use cases, that's
going to necessarily matter.
I think it's really for really intense stuff, like game engines, for example.
NICK WONG: Right.
Which does come up a lot.
COLTON OGDEN: Yeah.
NICK WONG: Game developers have to deal with a lot of things like that.
It actually matters.
All right, cool.
"C has not a garbage collector."
You're right, C does not have a garbage collector.
C has very little, actually.
COLTON OGDEN: Yeah, C is a very lightweight program.
NICK WONG: It's pretty limited.
COLTON OGDEN: Well, amazing libraries that people--
NICK WONG: Exactly.
COLTON OGDEN: Does Newt compute from one religious debate to another?
That's been sort of the theme today.
Everybody was shutting out David.
David asked about the tabs question.
NICK WONG: Yeah.
COLTON OGDEN: Pascal Case says, code editor, feels good man.
NICK WONG: Cool.
COLTON OGDEN: It's great in C+, but you go to Python, and people--
NICK WONG: Yeah, people--
[INTERPOSING VOICES]
COLTON OGDEN: Instant vomit.
NICK WONG: Yeah, they just can't deal with it.
I can barely deal with it.
I think in my C programs, I write in underscores.
COLTON OGDEN: [? TheDuckThough ?] says, ew, C#.
I actually like-- if you're comparing it to Java,
I actually like C# quite a bit.
NICK WONG: Yeah, I'd agree with that.
COLTON OGDEN: I tend to prefer coding in Python over C#.
C#, I actually enjoy.
I use it in Unity development.
It's actually a pretty nice language.
It's got a lot of cool features.
And they're consistently adding new stuff to it every other year,
which is great.
NICK WONG: Yeah, that's true.
COLTON OGDEN: Tabs are converted to spaces,
aren't they, asks [? Bhavic Knight ?] In the context of text editors, yes.
NICK WONG: Yeah, they can be.
You have to be careful about that.
Python will catch you on it if you're switching through things.
I'm sure a bunch of other languages, interpreters, will catch you on that.
COLTON OGDEN: Yeah.
If you you're doing just raw, like a raw text editor or something,
and you're using tabs versus spaces, it's not going to know the difference.
It's going to know the difference.
NICK WONG: Right.
They'll pick up on that.
COLTON OGDEN: OK.
Thank you all, from the low level programmers, says [? HeenaLar. ?]
Thank you very much for joining.
People want to fight religious debates.
NICK WONG: Yeah, every time you bring up like this sort of thing,
once you get into like C or C++, or actually any programming language,
I guess, people will always have their own opinions on like what libraries you
should use or shouldn't use.
I'm generally pretty lax about that.
The times that I would not be are when they affect memory or efficiency,
or they have some known bug that I'm aware of.
But otherwise, if it gets the job done and it does not
violate any immediate practice that I know of, then I see no problem.
COLTON OGDEN: Enough boring C. Let's do inline assembly.
NICK WONG: Inline assembly.
Yeah, I don't really want to ride assembly.
COLTON OGDEN: That's some 90s programming right there.
NICK WONG: Yeah.
COLTON OGDEN: [? Nowanda353 ?] asks Nick, what's
the best thing you've built so far?
NICK WONG: Oh, that is a great question.
I have built lots of cool things.
Sorry, that sounded very arrogant.
That was not intended that way.
I think I tend to detach myself from the things that I build,
and just kind of look at them as their own objects,
and think that they're kind of cool.
So I spent a lot of time developing a auto grading software for a club
that I run here.
There's some labs that I've built for a cybersecurity club
here that I think are very cool.
I built my own version of a VPN.
Originally, it was based on open VPN, and then
I kind of modified that and tweaked it.
So I guess I would say I kind of like stood on the shoulders of giants there,
but I thought that was really cool.
I built, or I guess I leveraged a Raspberry Pi Zero,
it was very small, and built or designed a device that
allows you to plug it into a computer through USB,
and it can simulate basically any kind of connection or hardware device
that you'd like.
So you can kind of pick through the Raspberry Pi's interface, which
generates its own Wi-Fi network when it has power,
and you can connect to it through, I built an app for it on my phone,
or you can just connect to it via the Wi-Fi interface.
What that lets you do is you can pick what you would like to simulate,
so you can simulate like an ethernet connection,
you can simulate a keyboard connection or even a mouse connection
if you'd like.
And I found that the keyboard and the ethernet interface are the most useful,
in that what I can do is then, if I'm simulating a keyboard,
then I can actually brute force like the password
to your computer, which is kind of cool.
I think that's awesome.
I would never use it.
I have no need to, but I think that's awesome.
You know, I got to go, this is a really cool thing to be able to do.
If you simulate an ethernet connection, then what you can actually do
is, I guess, pretend that this is now the internet.
You can set yourself as like the highest priority ethernet connection,
and then your computer will send out connections
because it's listening for and asking for like site updates and things,
even when your computer screen is locked,
so I can kind of bypass your password in that way,
and maybe poison a bunch of your browsers.
And then when you log back in, they are all like messed up
and they will reach out to me instead of where they're supposed to go.
COLTON OGDEN: White Hat Hacking for the win.
NICK WONG: Right. [? AllWhiteHat, ?] I actually do-- am very,
very solidly against hacking for personal gain.
I think that that is awful.
So there's all sorts of, I guess, ethical debates and qualms
that I have, or maybe not qualms.
I very thoroughly am against people who hack for personal gain.
I think that that would be awful, based on the sense that I
think hacking in computer science--
and in general, I have a pretty positive view of humanity--
should all be used to help people in general, and help the community,
help the world as a whole.
So.
COLTON OGDEN: It seems to be the case that, I mean, just
by knowing how it works, you have the best defense against that.
NICK WONG: Right.
Also true.
COLTON OGDEN: The thing that you hate so much, you find a way to combat it,
I think.
NICK WONG: Exactly.
COLTON OGDEN: And they're saying shout out for Google's future CEO, Nick.
NICK WONG: I appreciate it.
COLTON OGDEN: OK, we have done a ton of comment reading.
We'll pause here.
I'll come back on the comment reading.
Let's go to the next, I think, sort of C coding.
NICK WONG: Yeah, we'll keep going.
So we have now kind of checked our file descriptors and things.
I guess we've checked we have the right number of arguments.
We have checked that we have an actual file that has been opened,
and then we set up a buffer.
We read into that buffer and we print in the standard out for that buffer.
So one of the kind of cool things that you can do--
oh, whoops.
Not what I wanted.
Make sure that that's up to date.
And then what I can do is read in itself, actually.
And that broke, which is great.
I gave it three arguments because I can't read.
And we get the first 8 bytes from our own file, which is kind of interesting.
So what this allows us to do is, I can set this, we might say like some int,
and it's actually a little bit more general here, and we'll have N,
and we're copying in N bytes.
And what I can do is then recompile this and run this,
and I get that, which is kind of interesting.
COLTON OGDEN: Apologize also that [? MissSchick ?] followed us
at the same time that [? DuaneNeese ?] did,
so thank you very much for following.
NICK WONG: Awesome.
And so you get these kind of like weird characters
that my terminal does not interpret, and just kind of like
throws its hands up and gives up.
But generally, I have now been able to-- oh, whoops, that's not what I wanted.
I can copy up to 64 bytes of my file, my input file.
And I can do that with any file, really.
I could even go to like my desktop.
And I don't know what's on my desktop, but sure, we'll
do our like Bash history.
Oh, that's a bummer.
So it couldn't actually open that file, which is a shame.
Oh, [? .bashhistory ?] is probably why.
There you go.
And so you get the first 64 bytes of that.
So we're doing something very simple.
We're just opening a file, reading out the first 64 bytes,
reading it into a buffer, which might cause
people to think oh, this is terrible.
But that's OK.
We're going to kind of ignore that problem for now.
But what we could do after we have started to open files and everything
is, maybe we want to be able to read in the whole file
and then print it out in buffered chunks.
And that would be a very reasonable thing to do.
So what we're going to do is while--
and you might notice there's an unused value here.
Technically, what I could have done is this,
and if I had better warning setup, than I would have gotten
a warning on the fact that f.read returns a value that I'm ignoring,
but we're going to kind of ignore that for now
and pretend that it doesn't matter.
I don't know why I started typing up here.
I'm losing my mind.
And so what we can do is while f.read buff, 1 and N file is not equal to--
I believe it returns 0 in the file.
We'll find out.
Then what we can do is print out that buffer.
And what this, in concept, if I'm not losing my mind,
which we all know that's not true, that should
allow us to read in the entire file, and basically, first
is going to have a different name now.
We're going to call it Cat, and ./cat of first.c should read out--
pretty close.
We're reading out some junk.
We shouldn't necessarily be doing it that way, but that's OK.
So this is roughly the right concept.
Oh, there's a bunch of those.
Oh, OK.
That kind of makes sense.
Does that make sense?
Let me think about that for a second.
This shouldn't necessarily be junk there.
COLTON OGDEN: Have you got those semicolons?
Are those semicolons?
NICK WONG: It looks like they are all--
I don't know.
Let's see if we can figure out what those characters are.
COLTON OGDEN: Oh, no, they're not semicolons,
because there's a semicolon there in that.
NICK WONG: Yeah.
There's kind of an interesting character going on there.
COLTON OGDEN: I think you've been hacked.
NICK WONG: Yeah, I've been hacked by magic.
COLTON OGDEN: Clang.
NICK WONG: Yeah, that's an interesting character.
My guess is that that deals with junk that is not the null terminator,
would be my guess, but we'll find out, maybe never.
But that does roughly cat our file out.
I also know that this terminal has some bugginess, where if I type underscores,
they don't always show up.
Like I can I think echo, underscore, underscore.
Oh, just kidding.
OK, so you know.
COLTON OGDEN: Does your normal shell work?
Not the integrated one in VS code, but the one that's in your actual terminal.
NICK WONG: We can find out.
Let's see.
That is a great question.
Oh, well, you get that.
Let's see.
Where is everything?
./cat first.c.
Oh no, it's even weirder.
COLTON OGDEN: OK, fascinating.
NICK WONG: Interesting.
My guess is that there is some junk sitting in the buffer
and I'm not clearing it.
I don't actually zero the buffer.
And so that might be something to look at,
is like what happens if we fprintf--
whoops.
I love that we always-- this is like the perpetual sidetrack.
COLTON OGDEN: It's a good line.
NICK WONG: Yeah, it's like a very cool, I think, thing to be able to do.
This actually should-- well, no.
It won't [? segfault. ?] That is OK.
It's OK to type.
Now, if I do first--
actually, let me also--
.txt.
COLTON OGDEN: I was about to say, do you want to like
write a test script for it.
NICK WONG: Yeah.
And then we can run this, test.txt.
And OK, cool.
It ends up actually being OK as far as that goes.
My guess is that at the end, we are encountering stuff
where it's not reading exactly.
Because our buffer is 64, so let's say I read in 1024 bytes.
COLTON OGDEN: Oh, if it reads more than the number of bytes that are
in the file-- it's going to try to read--
NICK WONG: Yeah, my guess is it's doing something weird.
COLTON OGDEN: Because there's not 64 characters in that text file.
NICK WONG: Right, exactly.
So it's doing something kind of strange.
COLTON OGDEN: It's just reading.
NICK WONG: Sure, that's fine.
And so if I was smart and if I wanted to actually like thoroughly debug this,
then what I would say is, we should be very
careful about making sure that buff gets set to 0
after you've printed everything out.
So like you could do buff is equivalent to like null byte or something.
0 would actually possibly work at the end.
Actually, what I would do is something like this, an mcopy.
I don't remember how exactly an mcopy works.
I believe it's-- well, let's see if it--
oh, Visual Studio.
Tell me.
I was really hoping it would.
COLTON OGDEN: I think you can get that to happen
if you install the right extension.
NICK WONG: I know.
I really should spend more time installing the right extensions
on these.
Destination, and we'll say source, size.
COLTON OGDEN: [? HiFiCS, ?] thank you very much for joining.
NICK WONG: Actually we could just--
yeah, actually.
That's fine. [? Constantvoid*-- ?] we'll do null byte.
And let's see what that ends up doing?
Whoa, that's not good.
Oh, string.h.
Good to know.
And I actually generally prefer, I mean, generally I
prefer knowing which libraries I need.
But if I don't, then I would much rather be told what to do there.
No argument where nonnull is required.
Oh, that's a bummer.
Where am I doing that?
What is the null nonnull argument?
COLTON OGDEN: Does it consider--
It doesn't consider the null terminator character to be null, does it?
NICK WONG: I don't think so.
Oh, maybe it does, because there's only three arguments required.
Well, that's kind of cool.
So what we could do--
maybe that's just-- actually, what I could do is just null.
Let's see that does.
We'll find out.
Oh, it's even worse, but maybe it'll work anyway.
Yeah, there we go.
Get some even worse behavior.
COLTON OGDEN: Safe.
NICK WONG: Yeah.
Very, very bad.
So if I wanted to, I could play around with this and debug it,
and we'd be fine.
I am pretty confident in saying that like,
it's just junk at the end of the buffer.
COLTON OGDEN: [? TheDuckThough ?] is saying
memset buff 0 and then size of buff.
NICK WONG: Memset is a more useful command there that we could use.
Yeah, there we go.
So that's gross.
We'll do 40 or 64, and then, oh, well, another thing to consider
is that each of these buffers does not have a null terminator at the end.
And so that might be causing some of this problem.
We can do that here.
Memset, did he use the actual syntax from memset?
Buff, 0, N, probably.
COLTON OGDEN: Buff, comma, 0, comma, size of buff.
NICK WONG: Cool.
That works.
COLTON OGDEN: And in your case, as he says above.
NICK WONG: Cool.
My intuition is going to say that that deals with my null1 terminator.
COLTON OGDEN: Yeah, OK.
C is hard.
NICK WONG: Yeah, no, it's a pain.
Yeah, and that is what I will say is true,
in that the null terminator is not going on here.
So what you could maybe do is set this equal to the--
actually, yeah.
We're not going to deal with that too much because it isn't necessarily
the main point of the stream, but what I might recommend is setting everything
to null terminator, and then proceed with the way
that we're actually doing it here, and that should be fine.
Basically just copy in up to negative n minus 1.
COLTON OGDEN: [? Brenda was ?] asking, can you just
zero out buff before you read into it?
NICK WONG: Well, so actually, I believe what's missing
here is our null terminator, because you could do something like this
and I believe we'll end up with the same behavior.
Oh, crap.
That's a bummer.
No, there aren't.
Yes, there are.
I'm a huge fan of blaming the compiler until I realized what I did wrong,
and then I stop blaming the compiler.
I don't inherently see what's wrong with that, but--
oh, because I, sure.
We could do this, then, where you memset all of buff to 0 to N.
And I think we'll see the same behavior.
We can find out.
Yeah, we'll end up with the same behavior, and that's fine.
My very strong guess is null terminator dealing out here.
And it's OK.
We don't really have to deal with it inherently.
But maybe it's something to keep in mind.
Cool.
So after we've kind of dealt with all of those things,
we now have something that allows us to code kind of line by line, well, ish.
It allows us to take in buffers and things
and allows us to kind of go through our file one buffer length at a time.
However, it might be really reasonable to us to--
COLTON OGDEN: [? Oh, and Brenda ?] was saying your buffer is size n,
you're reading n bytes.
You don't have any room for the null terminator.
NICK WONG: Correct.
So what I would recommend doing is reading in n minus 1 bytes,
and setting the initial thing to the null terminator, or all of it
is to null, And that should actually end up being OK.
So you could do something like this, actually, which I believe would work.
Yup.
COLTON OGDEN: Wow.
NICK WONG: Now, there is a slight problem with that,
but that's OK, basically the problem being
that I have appended this whack n.
And so what you can do is here, and then we're good.
Thank you.
I thought about it and then didn't do it because I thought it wasn't worthwhile.
COLTON OGDEN: And [INAUDIBLE] is always like super help--
she helped me debug something that I was doing on stream too
at one point, a physics related thing I needed,
and I was like, what is going on right now?
NICK WONG: It's one of those things that are like, if you're
doing it on your own, never a problem.
You'll never face that problem.
But the second you do it--
COLTON OGDEN: It's only when you're doing it in front of somebody,
or a lot of people.
NICK WONG: Exactly.
Yes, if you're ever doing it in front of a lot of people,
they will immediately notice, and your mind will lose itself,
and nothing will ever work.
So thank you.
Appreciate it.
But now we might want to do something a little bit different.
We want to read actual line by lines.
I see someone, I think, [? MyAlCat. ?] IDK what he is doing.
Looks like trial and error.
And you are very much correct.
That is a lot of what this is, because we do code live,
so we don't actually prep anything, as much as I would love to.
Generally, when I do some sort of like seminar
or I guess lecture, general educational tutorial,
I like to prep like an entire script beforehand.
I like to prep a bunch of code and I like
to basically not have to really worry about coding in front of people.
However, in these streams, we found that it's
really cool to be able to kind of show that we're fallible too,
and we make all sorts of mistakes, and I'm actually
very comfortable making mistakes in front of people.
I do it constantly.
That's just my life, is just making mistakes after mistakes.
COLTON OGDEN: Same.
I've gotten used to it for the streams that I do.
NICK WONG: Exactly.
COLTON OGDEN: It's OK.
NICK WONG: It's totally reasonable.
COLTON OGDEN: The thing about it is it's very flexible.
Because if you're into a script and we want
to talk about some other random topic and you veer from your script,
you could screw your script up big time.
NICK WONG: Exactly.
So I like this actually quite a bit, and I don't necessarily
have any major problem with that.
COLTON OGDEN: [? MyAlCat, ?] we all make those mistakes.
Yes, we do.
NICK WONG: Yeah, exactly.
COLTON OGDEN: [? MyAlCat, ?] yeah, no.
NICK WONG: Yeah, no, we're not insulted.
Trust me, we are pretty much not insultable, I think.
COLTON OGDEN: I think so, yeah.
NICK WONG: We've been through plenty, and you're all good.
We've been on the internet before, and I mean,
I actually stream video games from time to time, and people are much worse.
I mean, you guys are super nice.
You guys had the chance to hack our WordPress website
last time, and did not put like porn on the WordPress.
I was like, you know, you guys are fantastic.
COLTON OGDEN: It could have been much worse.
NICK WONG: It could have been so much worse.
So we really appreciate the way you guys actually interact with us.
COLTON OGDEN: Now it would've been hilarious
and probably a meme.
It probably would've become a meme.
NICK WONG: We would've had to just cut it out, I think.
COLTON OGDEN: If he would have clipped it, though.
He would've clipped it.
NICK WONG: Exactly.
Our editors, I think, would have cut that one out.
So yeah, no worry.
You will very rarely offend us, I would argue probably never.
It would take quite a bit.
That's not a challenge like this.
Keep doing what you're doing.
That's good.
COLTON OGDEN: [? MyAlCat's ?] asking what's your Twitch stream?
NICK WONG: Ah, so it's pwrhausdacell It's
a modification of my like general kind of coding handle.
COLTON OGDEN: OK, I could write it for you.
NICK WONG: Yeah it's, P-W-R-H-A-U-S--
COLTON OGDEN: Oh, P-W-R-H-A-U-S?
NICK WONG: Yep.
D-A-C-E-L-L.
COLTON OGDEN: Is that correct right there?
NICK WONG: Yep, that should be it, I believe.
And that is my gaming twitch.
I like to kind of throw that out there.
It's a lot of fun.
[? The DuckThough ?] says porn it is next time.
No.
We would think it's very funny, and--
COLTON OGDEN: We're gonna lock that down.
NICK WONG: Yeah.
There probably won't be another next time.
Probably the only time that we'll ever let you guys just hack us.
Yeah.
So all right, we have all of this going on.
Something that you might be familiar with is fscanf.
And so we have file*, [? constant*, ?] and then our buffer.
Oh, man.
So we're going to do all sorts of things.
And you're reading this?
COLTON OGDEN: [? GoToPoopie ?] says, I do 20% coding, 80% debugging.
NICK WONG: Yeah, that sounds about right.
That might even be high on the coding.
I think I spend most of my time debugging, and I kind of like it.
Like I like the iterative process and kind
of like recalling things that I should have known before and stuff like that.
I think that's very good to know.
I'm trying to think, oh, a buffer size of 4
is probably not going to work there.
We'll say 1024.
That also might not work, and I have a inkling of an reason why.
Yeah, there we go.
I think that's hilarious.
Oh, so I'm going to stop it from running.
I always love when things like that happen, because I think
that they're actually quite funny.
So what I basically did was made a small change.
But if you go on to the man page for fscanf,
OK, well, my computer has decided to not work anymore.
Cool.
So we're going to wait while that one catches up to what's going on.
If you'd like, you're welcome to take some guesses
as to what went on, but basically, the return value of fscanf
is not quite the same as the return value for--
oh, great.
We're going to reopen that.
The return value for, what was I using, fread, I believe.
And so that is something to kind of keep in mind when you're actually-- oh, no.
Man, live streaming really does kill the ability to type.
COLTON OGDEN: [? RedandGray, ?] [? TheDuckThough, ?] and [? AsunaHQ, ?]
thank, all of you for following.
NICK WONG: Yes, thank you very much.
COLTON OGDEN: Oh, [? TheDuckThough ?] is say, guys, use size up, please.
And this [? nonconst int ?] makes my head ache by looking at it.
NICK WONG: Yes, so [? TheDuckThough ?] points out a very good point,
which is that I have been using n and they say [? nonconst int, ?] which
basically means that I am using this in order to talk about the size of certain
things.
And so what they would argue there is that size of 4 buff
would be a better choice, and I very much agree.
That actually makes perfect sense.
So totally reasonable to do.
I would argue in this case, it's relatively harmless
as far as differences go, but definitely a good idea.
COLTON OGDEN: Yeah, like for smaller program, not necessarily a big deal.
But if you're working on a big code base,
obviously these kinds of decisions make sense.
NICK WONG: Right, and I think it is actually a good practice
to kind of hold yourself to that sort of extensibility problem,
where if I were to move this variable to like some sort of configuration file
and it's not necessarily immediately accessible,
and I used it maybe a bunch of times instead of just one, so in this case,
I use it twice.
Oh, wait, no.
That's a terrible idea.
There we go.
Then it would actually be much worse to do.
I also realized a bunch of you guys are saying you can't find my stream.
I might've misspelled it.
I will double check on that and get back to you.
I don't usually like play any video games when I'm here in college.
I only play when I'm back home, so basically, I
don't really memorize it super well, but there we go.
So now we can go back to what this is actually doing, which is fscanf,
or scanf is actually what we're probably going to use.
Sorry.
Although, fscanf in this case--
oh, actually, fscanf works better in this case.
I was forgetting that we aren't scanning from a stream.
We want to go all the way down to return value,
and this will point out to us what's going on exactly
with why this doesn't work.
I believe it returns null upon reaching EOF, but we will see in a hot second.
I always-- well, there you go.
Scroll right past it.
There you go.
So we can actually say, wow, that is not equal to EOF.
And now we can recompile our program--
sorry, the colors are not quite as pretty--
and hopefully cut everything back out.
Now the reason that this is a little bit interesting
is that I actually stripped us of our new line.
So basically, what I ended up doing by putting new line
here was I pulled out-- oh, I hovered over it
so you can see what I'm talking about.
I pulled that new line out of the string and then printed it.
So that's why you get kind of this weird blend.
If I refix that--
oh, cool.
Then we'll end up--
oh, maybe I didn't recompile that.
I'll clear this.
There we go.
Oh, well, that's a bummer.
Oh, I guess it strips it out on its own.
That's kind of good to know.
Didn't know that.
Ah, there we go.
Bummer.
So that then pulls in the entire buffer, and I guess our buffer is pulling in--
oh, sure.
OK.
So then it tries to pull in all sorts of interesting things
with how it actually decides where a string starts or stops.
So we can do stuff like this, and I believe that will fix that error.
Oh, maybe not.
Just kidding.
That's not what that's going to do.
Oh, well.
So that is kind of the right idea.
Somone asks, is that function safe, and what you mean by that, I would assume,
is does it protect us from buffer overflows
where we actually like insert too much material by a little bit
too much into the buffer that we're actually using,
and maybe return to some other function start address.
And I would argue I don't believe fscanf is safe, at least not from that,
though there are a lot of functions that are,
and it's worth your while to look up which ones are which.
I generally forget because I don't build that much in C,
But it is worth looking at.
And something in general to consider when you're doing, what do you say,
if you're doing any sort of user input interaction,
then you want to be very careful of what we would call undefined behavior.
So what [? TheDuckThough ?] points out is
that this is a straight up buffer overflow vulnerability.
I thoroughly agree.
Do not do this in order to interact with anything that is like client facing.
If you're doing it to interact with yourself,
and it's a small script that you're writing only for your own use,
then I would question why you're working in C.
But technically, that would be fine.
It is something very much you want to keep in mind,
and it points to something else that we are kind of seeing here,
where we're spending a lot of time kind of dealing with how, in particular,
to do each of these things--
Sorry, where was I going with that?
Ah, how to kind of like nitpick and go through each individual piece
is very much worthwhile, and you can spend
a lot of time kind of really getting into each one of these steps.
I would argue that in general, any one of these steps
has something about it that we could talk about and maybe optimize or fix,
but there is something to be said for balancing it with what you are actually
trying to do.
And so in our case, we don't necessarily care about how any one of these--
I guess it won't really affect our end product,
which I forgot where we were headed with that, but that's OK.
It's fine.
We will now actually talk about making a makefile
so that I don't have to keep typing this whole,
like going up two commands and then rewriting stuff.
And that's something that's kind of just worth looking at,
and I promised that we would look at a little bit of assembly,
so we're going to write a much simpler file that is going to copy things
over one byte at a time, and that'll give us
something a little bit easier to look at as far as what we're dealing with.
So what we're actually going to do is rewrite this a little bit.
I believe it is just read as the syscall.
So this is not using standard IO.
Oh, this is using standard IO, but it's just read on its own.
And I think it takes in a file descriptor, actually.
Let me check.
COLTON OGDEN: I don't remember if I thanked [? RagashSharma ?]
for following, but thank you if so.
If not.
NICK WONG: Very cool.
So what we're actually going to do is modify this program a little bit
in that we're going to deal with syscalls rather than each of these.
So we're going to have an int file descriptor is equal to open.
I think this takes in the same arguments.
We'll find out.
Man open, man 2 open.
[? PatName. ?] Cool.
And what this is going to do is take in our v1, which we know already
exists, and its flags are--
oh, right.
It takes in int flags.
I believe read only.
[? 0 RDONLY. ?] There you go.
Read only.
I don't remember where those flags came from.
Oh, probably FCNT.
Right, that's what someone pointed out earlier.
COLTON OGDEN: FlyingCircles, thank you very much for calling.
NICK WONG: .H. I believe that has this flag.
COLTON OGDEN: It's not an underscore.
NICK WONG: Ah, read only.
Sweet.
That's what I like to see.
So we're now going to actually open this file and get a file descriptor.
Generally, if you wanted to do actual error handling,
we are going to like check for return of negative 1.
The standard for system calls is when you have some sort of problem,
you return negative 1.
I'm going to kind of omit error handling because I
don't want to run out of time, and actually talk about each of the things
that we want to get through.
COLTON OGDEN: [? MeowCat ?] makes a good point.
Syscall being equal to a system call.
NICK WONG: Ah, yes.
Sorry.
So, yes.
Oh, [? MeowCat. ?] Oh, I understand.
I was thinking [? MyAlCat, ?]
COLTON OGDEN: It might be.
NICK WONG: I think you generally nail the pronunciations a lot better
than I do, so I'm going to go with yours.
So yes, [? MeowCat ?] points out that a system
call is the programmatic way in which a computer program requests
a service from the kernel of the operating system it is executed on.
That is a very good point, and what that ends up actually meaning is--
what was I was going to say?
We have some sort of protective control transfer
from user land, which is where all of our processes
actually run, into kernel land or kind of the kernel space, which is scary,
and all sorts of dangerous things can happen there.
You have basically full privilege over a computer.
So that's what we are0 talking about anytime I say system call,
and the reason that I want to kind of switch us over
into using direct system calls is just so
that we can actually see a little bit more low level what's exactly going on,
using programs like strace and mmap.
Otherwise, you wouldn't necessarily need to use any of these,
unless you were dealing with this particular kind of use case.
COLTON OGDEN: [? TheDuckThough ?] just said protip, use mmap,
is that what you just said?
NICK WONG: Right.
So [? TheDuckThough ?] says, protip, use mmap.
With that syscall, you can directly map the file into memory
without needing to read it into a buffer first.
That is a great point, although not all files or file
descriptors are mmappable.
There are some which cannot actually be mmapped--
into memory directly.
And mmap has some problems that you might
face if you were trying to maybe, for example,
implement your own buffering system.
So standard I/O uses a buffer cache, which it basically says, and we'll
get into that in a second, it allows us to speed up
kind of whatever is going on with file read and write
or input/output, standard I/O. And that buffer hash is, I think, 4,096 bytes.
And what that does is, anytime you want to write something to file,
it's actually really expensive to write something to a disk directly.
And what I mean by expensive is that you have
to do some sort of control transfer.
I have to give power to the kernel.
The kernel's got to save state for whatever it was previously running on.
And then it's going to actually copy things over
into our disk, or solid state drive or wherever
we are actually storing things.
And that's pretty tedious.
It's pretty intensive on your kernel.
So instead, what happens is before we ever
actually pull some sort of protective control transfer
and deal in kernel space, we accumulate a bunch
of stuff in the standard I/O case.
You actually accumulate 4,096 bytes.
And then when you've hit that limit, or when you have kind of closed the file
descriptor and you want to get everything out,
then it will take that whole buffer and flush it to actual memory.
That's for writing.
If you're reading, it works kind of the same way
but in reverse, where when I'm reading a bunch of bytes from memory,
it actually is to my advantage to read a bunch of them at once,
even if I'm reading them one at a time.
We're going to see that in a hot second, where if I try to read one at a time,
I again have to do these protected control transfers back and forth.
And so what ends up happening is it's going
to be a lot slower if I read one at a time as opposed
to reading everything into a buffer and then blowing it through.
COLTON OGDEN: All right, cool.
Michael Abrash is a very famous game developer and worked on Quake and Doom
in the 1990s with id Software.
NICK WONG: Oh, sweet.
COLTON OGDEN: And he wrote he wrote a book called,
I think it's the "Black Art of Graphics Programming,"
and his first chapter is about optimizing a sort of speller program,
kind of, like look at a list of words, and instead of loading eight bytes,
like loading 1024 or 2048 bytes at once is like two orders of magnitude faster.
It was on a Dos platform.
Like back in the day, that made a difference between like one second
and like 30 seconds long for your program running.
NICK WONG: That is crazy.
COLTON OGDEN: It's like exactly what you're talking about.
NICK WONG: Yeah, no.
There's enormous performance improvements
that can be gained by using buffers efficiently.
COLTON OGDEN: Syscalls are expensive, is basically the--
NICK WONG: Yeah.
Syscalls are very expensive.
COLTON OGDEN: [? AxAJaw, ?] thank you very much for following as well.
NICK WONG: I'll say greater than 0.
All right, and then what this should allow us to do
is, then write, which is also a syscall.
I believe it takes a file descriptor, a buffer and also [? sum ?] size.
Oh, well, OK.
This will fail if N is not 1.
And N being 1 is actually OK, but this is OK for us.
And then we are going to close file descriptor.
Cool.
So then what we are also going to do is touch a makefile,
and we're going to do with that makefile.
So makefiles are really cool.
We're not going to talk about it too much, but I'm going to kind of point
out what everything does because it'll allow us to simplify
some of the other things we're doing.
So let's say that we want to just make cat.
So this is kind of an interesting--
We'll call it like, Democat.
That' kind of cool.
And that requires, we'll call it cat.c.
Oh, no.
Thank you, autocomplete.
And then makefiles, after you give it some sort of like pattern or keyword
that it can build, they also require like what command is actually
run so we'll say dash o is cat.
Dash optimizedflags we'll set to 1, and then cat.c.
And this should be-- oh, oops, sorry.
Democat, Democat and Democat.
That way we can kind of distinguish it from the actual call cat.
And then what we can do is also say for clean, I don't know,
usually you would remove object files and other headers and things like that,
but we're going to kind of just leave that as very simple.
Oh, sorry.
We don't need the R, but we need the F. Well, I like having the F there.
And cool.
So what I can do with this is I do make clean.
Democat doesn't exist, but it would have run it.
We're also going to move first.c into Democat.c.
Cool.
And we're going to, I guess, remove cat as well, remove first, and remove--
actually, test.txt is fine.
So now that I've run make, it should-- oh, nice.
I have a bunch of implicit declarations of these functions.
I really thought they were part of standard library.
They might be part of like sys.
We'll find out in a second.
Let's do man 2.
Read would work.
It doesn't really matter, as long as it tells me which library.
Oh, whoops.
That's a library we need.
We might want to remove--
[INTERPOSING VOICES]
Yeah.
We don't need a bunch of these libraries now,
so we're actually going to kill a bunch of them.
I don't think I need those.
COLTON OGDEN: Everything breaks.
NICK WONG: It'll all break now, which is great.
You need estd.h.
Cool.
And I believe that has them now.
Returning a return value of right.
Sure.
COLTON OGDEN: [? Brenda ?] was saying that she, I
think she made a reference to using Uni standard earlier.
I must have missed that.
NICK WONG: That's probably correct.
If you guys brought it up, I'm sure you did.
Generally, you guys bring up-- oh, where am I missing a return value of right?
Am I ignoring it somewhere?
Oh, OK.
So apparently, casting that to void does not work.
Sure, I'll use it.
There you go.
I have used your return value.
Cool.
And then when we run Democat, if I don't pass on the file,
it should tell me that I didn't pass in a file,
and then I should also be able to--
oh, test.txt, and it did nothing.
Yay.
Oh, right.
It did actually do something.
I just forgot what it was doing.
So we read everything and then we wrote it back out, which is kind of cool.
So this is actually interesting.
That did something kind of weird.
COLTON OGDEN: I read that as Democrat.c for a second.
NICK WONG: We're running Democrat on its own.
COLTON OGDEN: [? DiscoDamage, ?] thank you for following.
NICK WONG: We're going to actually write fprintf standard out of %s buffer.
And that should allow--
whoops.
Not what I wanted.
There we go.
democat test.txt.
Cool.
So we are now catting everything out by one character
at a time, which is very interesting.
So someone asked, why are we not importing standard library,
or like standard I/O I think I actually do import both of those,
but I'm not using them at the moment.
And the reason for that--
actually, I am using standard I/O. Whoops.
I needed that.
The reason for that is that I want to put us at a slightly lower level
than those libraries would put us.
Because those are libraries have already kind of created their own versions
of each of these things.
We were using them earlier, where you have like fopen, fclose, things
like that.
I'm trying to have us deal a little bit more with very low level, actual--
pretty much as close as we can get to syscalls.
Sorry, syscalls themselves.
I don't know why I said as close as we can get.
This is the closest we can get.
They are them.
This is exactly what we're talking about.
And the reason for that is I want us to be able to see a little bit more
about how each of these things actually interact with our computer.
So we're going to deal with something called strace.
strace is-- I think it stands for trace syscalls.
And we're going to strace.out as our output,
and then we're going to run democat test.txt.
And strace is going to write that to strace.out.
Most of these beginning ones, you'll see mmap is a syscall that's run up here,
and there's a bunch of other stuff.
I think it's like line--
oh, there we go.
So the first line started--
COLTON OGDEN: These are all syscalls.
NICK WONG: These are all syscalls.
Right.
So the cool thing about strace is it actually
tells you exactly which parameters were passed to the syscall
itself, which is very cool.
And it tells you its return, which I think is also very cool.
So if you're ever debugging something where
you're working at a very low level, then this is particularly useful.
So what we're looking at is openat--
sorry, I'm going to ignore the open.
That is technically what our open does, but we're
dealing mostly with writes and reads.
So our reads here are exactly what are going on.
You'll notice that there's this write system call.
And if you look in our code, we don't have a write system call.
None actually exist.
You might think, well, I thought we were talking about only system calls.
And I will point us to this function, fprintf
which does not actually explicitly tell us that it uses a syscall.
However, in order to write to console, you do actually need a system call.
So one of the great inventions of the Linux system
is that you have everything as a file, and as a file descriptor.
So I can describe the console, and basically
standard in, standard out, standard error, all as files.
In fact, they all have their own standard file descriptors.
There is a variable, I believe, that is like that particular file descriptor,
but we're not going to necessarily deal with that.
I happen to know that it's 1, off the top of my head.
And so I can actually now write to this file descriptor buf and n.
COLTON OGDEN: That's actually what somebody said.
[? TheDuckThough ?] wrote that in the chat there.
NICK WONG: Nice.
So [? TheDuckThough, ?] you kind of skip ahead, I guess, and that's very cool.
Yeah.
Now we can explicitly tell what's going on in our write system call.
However, if you'll notice in this original strace.out,
there's only a single write system call.
That's going to change.
So what's going to happen is, if I recompile this--
oh, right.
I have make now.
Oh, that's disgusting.
What?
Got to love the warning output.
COLTON OGDEN: [? Bhavic Knight ?] says, "This is
starting to get a little overwhelming.
What are we trying to do?"
If you want to maybe reiterate the goal.
NICK WONG: Sure.
So basically what I'm going to talk about
is we are building a version of Cat, which is a kind of basic Linux commands
or bash commands that allows us to take a files contents
and just print them out to console.
And then what we're going to do from there
is see if we can kind of consolidate some of these read
and writes a little bit.
We're not going to go all the way into that, because it
would take a little bit too long.
But the basic concept is pointing out that what's
going on underneath the hood is actually really, really important.
And so it's kind of useful to us to see that, even though we are working in C,
you can go at different degrees of high or low level,
even within a language that is notoriously low-level.
Hopefully that clarifies.
COLTON OGDEN: So taking it from like standard I/O to using syscalls
to write using the kernel.
And maybe eventually Assembly, if you have time.
NICK WONG: Yeah.
COLTON OGDEN: I'm not sure what your plan is for that.
NICK WONG: Exactly.
So I guess, to go back to the comment that was "libraries are for the weak,"
we're now basically ignoring standard I/O.
However, I am apparently implicitly declaring exit,
and I really don't feel like dealing with that,
so we're actually going to switch to return.
And that's fine for now.
So I'm going to get rid of standard I/O, because we don't need that anymore.
And that failed.
Where do I-- oh, right.
Right.
This isn't actually going to print anything to us now,
it'll just return 1.
We're kind of scrapping a lot of stuff, so that things
are a little bit cleaner.
So basically, now our code is all system calls, which is kind of interesting.
And what we're going to hopefully see is that, when we run our code again,
our strace output should change.
So I'm going to bring us back to this strace output.
And what I'd like to point out is that Hello World got written as one system
call, and bye world also got written as one system call of 13 and 11 bytes,
respectively.
However, that's going to now shift, because now, we
write out each individual--
COLTON OGDEN: Character.
NICK WONG: --character.
And with a syscall, that means that it has to be done individually.
And now we get what we would expect.
We get this kind of interleaved read a character, write that character,
read a character, write the character, read a character, write the character.
COLTON OGDEN: Same result, just a lot less efficient.
NICK WONG: Exactly.
It's so much less efficient.
And so if you're wondering, why 1?
There's actually a standard set of file descriptors for the first couple.
The first one is standard input--
sorry, standard output-- sorry, standard input.
I'm doubting myself.
Standard input is the first file descriptors.
So that's 0.
Standard output is 1, and then standard error is 2.
And those first three are always going to be that way,
unless you do something crazy.
And so that's what we see ending up happening here.
Now, something that can be a little bit interesting
is, if we were to do this on something that's really, really large, then
that might actually be kind of problematic, because that file is
going to be read very, very slowly.
So let's say we wanted to cat out, using our democat program, something
that was extremely large.
So let's take a file.
Let's do yes, Hello World.
I should put that in quotes.
Yes is a program that just repeats things.
COLTON OGDEN: [? TheDuckThough ?] pointed out,
if we wanted to be clear with the number,
we get to use the function fileno, and take an stdout.
NICK WONG: Right.
Fileno would also work.
And I believe there is actually a library that declares those variables,
and it's something like standard out fileno.
Yeah.
That would also work.
Actually, I would argue that this is a much better system.
And generally, you'll want to do that.
For the same reason that we would use like sizeof buf instead of n here.
Although, I am using n.
And there is no good reason for that, at the moment.
That's a terrible practice, and I'm not following my own advice.
So we will clarify that here.
COLTON OGDEN: [? TheDuckThough ?] will be very pleased.
NICK WONG: Yes.
And I thoroughly agree.
So yes, Hello World is going to print out a bunch of Hello Worlds.
And what we can do is, let's say into head n--
we might need like 10--
is that a million?
Yeah.
We're going to pipe that into test.txt.
Broken pipe.
Cool.
And that makes sense.
I generally would recommend using more of a while loop.
That's got to be an enormous file, we'll see if it can actually load.
Nice.
It freaks out.
And we should have a million lines of Hello World.
If we scroll all the way down just for funsies,
we see we get to a million lines of Hello World.
Yay.
So we have now built a test.txt, and if I do democat test.txt,
and redirect that into dev/null, you'll notice that it takes non-zero time.
And if we wanted to get a real metric on that,
we could actually time it, which is kind of cool.
So you know what that run and see what happens.
COLTON OGDEN: [? 42Force, ?] thank you for tuning in.
Says "Hello, Colton and Nick.
NICK WONG: Oh, that's awesome.
And we go through, and it took us eight seconds, which is such a pain.
If we look at Hello World and see H-E-L-L-O comma space W-O-R-L-D--
I think I have an exclamation point.
So we're at 12 right now.
Oh, no, I do not.
And then we get to 13 as our size of the string that we have.
We repeated a string a million times.
So we have 13 million bytes in our file.
COLTON OGDEN: And they're translating that to 30 million syscall, right?
NICK WONG: Yes.
[INTERPOSING VOICES]
So there actually--
COLTON OGDEN:
Well, the new line is one, [? right? ?]
NICK WONG: --26 million syscalls.
So that includes--
COLTON OGDEN: Oh, because the read and write.
The read and write.
NICK WONG: We're reading and writing, which is awful.
So what we can do is if I actually cat test.txt, which is kind of funny,
you'll notice this is noticeably faster.
We have exactly 13 million things going on here.
We have 13 million characters, which means 13 million bytes,
for our purposes.
And you'll notice cat is--
substantially faster is an understatement.
It's much, much faster.
COLTON OGDEN: Many orders of magnitude faster.
NICK WONG: Yeah.
You can do a basic time on cat, and it's not even close.
COLTON OGDEN: Almost zero, complete zero.
NICK WONG: Yeah.
For our purposes, it's zero.
So that's kind of crazy.
So what we might want to do is see if we can get a little bit closer
to the actual syscall that's going on.
And we know through our strace output-- oops--
that we have this kind of ridiculous interleaving of
reads and writes going on, and that's horrifying.
COLTON OGDEN: Best choice ever made.
NICK WONG: Yeah.
COLTON OGDEN: [INAUDIBLE] but best system program ever made.
NICK WONG: Best system program ever made.
You get to wait for hours and hours, if you
want to copy over a reasonable file.
So a reasonable suggestion might be, OK, well, let's double the size of it.
So it should go substantially faster.
COLTON OGDEN: [? UnsignedEd ?] suggested,
"Change your end to be 4,096.
NICK WONG: Sure.
Let's change it to the size of the standard I/O
buffer, which is 4,096 bytes.
COLTON OGDEN: Is that defined as a constant in a library, as well,
you think?
NICK WONG: I believe it is.
It is something that is used by standard I/O.
We're ignoring the return value of write.
We're going to just pretend that that's OK.
We would actually make use of these read and write return values,
if we were to set up our own actual buffer.
Now, in this case, it doesn't necessarily matter too much to us,
but we will get some garbage output, I believe,
if we don't set this perfectly.
I think this will actually trigger some weird output, which is fine.
So we run make.
And everything's up to date.
Now, if we run time of Democat test into dev/null, way faster.
COLTON OGDEN: Wow.
NICK WONG: Substantially faster.
COLTON OGDEN: That's faster than Cat, isn't it?
NICK WONG: It really is.
Yeah, I actually think it was way faster.
COLTON OGDEN: Cat's twice--
NICK WONG: About half as--
COLTON OGDEN: You've made a better version of cat.
Wow.
NICK WONG: Half speed.
Well, so correctness-wise, we're actually not correct.
If you were to cat or democat--
here, we can actually do this.
Democat this into test2.txt and then we do diff of test.txt and test2.txt.
They're not quite the same.
COLTON OGDEN: Interesting, it has h at the very end by itself.
NICK WONG: Yeah.
That's a that's a pretty big bummer for us.
That might be something--
COLTON OGDEN: Oh, it is because the 4,096 at the very end,
it can't read that 4,096 bytes?
NICK WONG: So basically, it ends up being that, let's say we have 4,096,
so the nearest multiple of 4,096 that's under 13 million--
I don't know off the top of my head.
COLTON OGDEN: So it just won't call the fractional part
at the very end of that.
NICK WONG: Right.
It'll end up reading all of the bytes, so we know that that part's correct.
But what it writes is actually a buffer that has not been cleared.
So there is a part of that buffer that has extra bytes in it from before,
and that's actually not quite right.
So if we look at test2.txt--
which is where we redirected everything from our democat--
then if we cat test2.txt--
and we can trust that cat is correct.
I would argue that you're pretty reasonably--
COLTON OGDEN: Probably, yeah.
NICK WONG: --there.
You'll notice it has some extra lines in it.
Because of you cat test.txt, we have exactly one million lines.
But test2.txt has a couple of extra ones.
And those extra lines--
if we do 55 times 4,096.
I'll pull out a calculator because my brain is fried.
It's been a long day at school.
COLTON OGDEN: If you can do it normally in your head, that would be pretty--
NICK WONG: That'd be pretty cool.
I don't think I can.
Then basically, what you'll end up noticing is that that's around--
COLTON OGDEN: 225,000?
NICK WONG: Yeah.
So we get 225,000, and that basically means
that we are inserting 225,000 extra bytes at the end of our cat program.
Not good at all, actually.
And so something else that we can do is, let's say,
well, 8,092 should be a reasonable next double for our n program.
And we can remake.
I'm going to just stop searching through history.
Remake.
It's going to tell we're ignoring write, and that's fine.
And then what I'm going to do is time its output into test2.txt.
And this was actually a little bit slower,
which is kind of strange, because I would
have thought that, as we doubled n over and over again,
I should get faster and faster, because my buffer is now enormous.
And I'm certainly not below 8,092, I have 13 million bytes in there.
So we're not exactly sure what's going on,
and we might look to something like strace to find out.
So let's go ahead and find my strace.
And we're actually going to-- oh, right. test.txt.
I'm going to just wrap that into dev/null, because I don't really feel
like having that put out to my screen.
So we're going to strace.out, and you'll notice that what's going on here
is we are doing this explicit write of 8,092 bytes every time.
And we go all the way through, and that's exactly what's going on.
Things are kind of shifting around, and everything's working seemingly.
And if we run this again, but we remake--
so that we go back down to 4,096, and we do make, and run that,
and go back here, same thing.
No immediate reason as to why we would have something
actually be a little bit slower when we ran it with 8,096.
COLTON OGDEN: It can't write more than 4,096 bytes, because you said
that's the max size of the--
NICK WONG: That is standard I/O's buffer,
but we're not using standard I/O. We're doing our own system calls.
And so this is where one of my professors
likes to point this out as kind of a comical point.
Which is that you should do a lot of trials anytime you run stuff.
COLTON OGDEN: I feel like that's science.
NICK WONG: Yeah.
It's generally a good practice.
We might have had just a weird fluke.
So we ran a couple of trials.
They're all on the order of 0.00567, something like that.
So now we've remade this, and we're going to keep running this.
And you'll notice, this is also that.
So then let's double check.
4,096.
And you'll notice that that might have just been a fluke.
And actually, it wasn't a fluke.
There was something I did deliberately, but very sneakily,
that caused that difference.
And that difference was that I required the creation of a new file.
I did something like test2.txt, and that was that extra overhead.
Because when I went to redirect into a file that didn't already exist,
it had to create it, had to open it for writing,
and then it had to send stuff to it.
And dev/null does not behave that way.
So we end up actually being much faster if we're writing
the dev/null, as opposed to this.
And that performance change can be seen right there.
You get back to that 0.027.
COLTON OGDEN: Makes sense.
NICK WONG: So actually, in concept, if you went up high enough,
this should get very, very fast.
Now, you do have a good point in that there is a buffer
that eventually this doesn't work for.
There is a buffer size in which we eventually overmatch the buffer size,
and that can no longer be done.
COLTON OGDEN: Is that the amount of addressable memory?
Or sorry, not the amount of addressable memory--
NICK WONG: That would also be a limit.
You can only hold so much in memory.
COLTON OGDEN: Does-- every application gets its own limited amount of memory,
right?
Or is it only with--
NICK WONG: It should.
COLTON OGDEN: --32-bit OSes if they have the 2 gigabyte?
Because 32-bit operating systems only have 2 gigabytes of addressable memory,
right?
NICK WONG: There's actually a command to view how much memory
is allocated to any one process.
I don't know it off the top of my head.
Dang it.
There is a command to let you see how much memory any process
should be allowed to have.
You can up that memory.
If you wanted to be really feisty, you can up it to the entire installed RAM.
Wouldn't recommend it, but you could do it.
However, you're probably not going to hit
that before you hit the size of the other caches between us and the disk.
Because the disk itself has a cache that deals with the actual
writes to like some spinning hard drive.
And we will, generally, hit that, but we won't
see it strace, because strace is just telling us what the actual syscalls are
being called us.
And we can force it to call them however we like.
However, there is some point at which you will not get a performance
benefit to just doubling this n.
So we're going to keep it kind of low, at like 4,096.
But otherwise, that is a very good point.
COLTON OGDEN: [? TheDuckThough ?] is making a point about branch prediction.
And I thought branch prediction--
NICK WONG: Ah, yes.
COLTON OGDEN: --was in the context of just a single executable,
but it looks like they're making a point about it running the same program over
and over again.
Is that the case, where a branch prediction--
NICK WONG: Right.
So that one depends a little bit on who developed your processor.
Because I have also interacted with them mostly in the case of over one program
is runtime.
It learns throughout that program.
Something that is kind of interesting about branch prediction
is it is roughly the cause of like bugs like meltdown and spectre,
is because branch prediction kind of opened up
this opportunity for certain timed hijackings, which is kind of cool.
So if you're ever interested, there is a really cool paper on meltdown,
I believe, and it basically walks you through how it's supposed to work.
And then I see at the bottom there-- ulimit, thank you.
I wanted to put unameit, and that's not right.
[? ulimit ?] tells you all sorts of things
about how things are suppose be allocated,
and you can set those things, as well.
COLTON OGDEN: I've used that to change the number of files
that I could have addressed by one process at one time.
NICK WONG: Oh, very interesting.
COLTON OGDEN: Or, I guess could opened by one process at one time.
But I didn't realize it did other things as well.
NICK WONG: It does all sorts of things.
There's crazy amounts of customizability in ulimit.
COLTON OGDEN: Interesting.
NICK WONG: And then there is some discussion in our chat about the kernel
being in the stack.
That would be kind of an interesting question, in that each process gets
its own stack, threads have their own stacks,
and the kernel has its own stack.
And they all live in memory.
So stack is actually something that's specific to a process, roughly
speaking.
And then the kernel is in the memory, if I'm not mistaken.
Yeah.
The kernel's code lives on in some sort of disk,
unless you are certain operating systems.
I believe piCore and Arch Linux?
Not Arch Linux.
There's a different Linux distro that runs entirely out of memory.
So in that case, what I said about it's like code living in disk is not true.
Well, I guess it is kind of true, but not relevant to the computer.
You could run the computer with no hard drive, which is crazy.
But yes.
Almost everything that you're going to be dealing with is in memory.
So cool.
Let's see.
There's a bunch of stuff going on in the chat.
Always love the discussion.
COLTON OGDEN: Yeah, there's a lot.
We've been not reading quite as many comments
[INTERPOSING VOICES]
NICK WONG: Yeah, just so we can kind of get through some stuff.
COLTON OGDEN: Maybe we didn't read quite enough today.
[? ISOTV ?] was saying that they need to leave early, so thanks so much,
[? ISO, ?] for joining us.
NICK WONG: Thank you for joining us.
Yeah.
All right, cool.
So we have now written some of this.
So maybe we want to implement a little bit
of our own kind of buffering system, because we've said that correctness
is really not there at the moment.
So what we might want to do-- and we're going to kind of adjust
this a little bit, so that maybe our write system calls are all granular,
but our read system calls are totally fine.
Well, actually, we might run out of time before that is even done.
So let me think about that.
Sure.
So we'll do some version of cat that will improve our reading system calls,
but not necessarily our write ones.
So let's say that we're going to create some sort of new function int write--
I will say demowrite, since we're being consistent.
And what demowrite requires is some sort of int file descriptor, as well as
some void star--
actually, we'll just say char star buffer, and some size_t n.
And we want it to write out some number of bytes to that file descriptor,
but we're going to just have it do that one byte at a time.
So for int i equals zero i is less than n, i plus--
you could do this in one line, but for readability, I'm
going to put it all in here.
We're going to then call the system column, which is
write of a file descriptor and buffer--
oops.
Buffer 1.
I'm sorry.
And this will be buffer at i.
So this will write individual characters.
And we should be good on that.
So let's go ahead and verify that demowrite now works.
Let's see if we can cat.
Oh, whoops.
Make first.
We get some errors.
Of course there's errors.
COLTON OGDEN: DLF1010, thank you very much you're following.
Or just DFL10, if we're reading it from binary to decimal.
I believe 1010 is 10.
NICK WONG: 1010--
COLTON OGDEN: Because it's 2, 8.
NICK WONG: 2 and 8, yeah.
Funny.
COLTON OGDEN: DFL10, then.
NICK WONG: Oh, wait.
That's kind of great.
10, 10, and 10.
Wow.
I had never really thought about that.
COLTON OGDEN: Now you'll never forget.
NICK WONG: Yeah, that's awesome.
It's just something that you don't really think about that often.
COLTON OGDEN: [? TheDuckThough, ?] thanks
for tossing in some resources on how OS works.
He was talking about BIOS, bootloader, kernel, [? UserLand ?] programs.
NICK WONG: That's awesome.
COLTON OGDEN: He posted a link, osdev.org, it looks like.
NICK WONG: Sweet.
COLTON OGDEN: [? TheDuckThough ?] sounds like they have a bit of experience
with systems and OS program.
NICK WONG: That is fantastic.
I think the more experience you get with this sort of thing, the better.
COLTON OGDEN: Maybe even more.
A lot of this is somewhat new. [? Syscalls ?]
aren't new, but a lot of sort of looking at this as interesting.
NICK WONG: It's awesome.
So yeah, we have now built demowrite.
I believe we still ignore that return, but that's fine.
I'm really not a huge fan of having warnings,
so we're going to just say int r, and then we're
going to just cast a void for r.
And that's not a great practice for actual development.
If you were doing production-level development, this would be terrible.
But we are kind of just going after the concepts,
and so it's close enough, for what we're trying to do.
And now, we want to make sure that that actually still works.
So let's then just democat itself.
And we get gibberish.
COLTON OGDEN: That's great.
NICK WONG: Yeah, let's beautiful.
COLTON OGDEN: Ship it.
NICK WONG: Go.
Let's see what that does.
And so this is nothing along the lines of what we want at the end here.
That's disgusting.
However, the things up here, probably fine.
COLTON OGDEN: Are you still reading 4,096?
NICK WONG: We are.
COLTON OGDEN: And the program isn't that long.
NICK WONG: And this program isn't quite that long.
COLTON OGDEN: So it's all garbage.
NICK WONG: That's all garbage.
COLTON OGDEN: [INAUDIBLE] memory past the point.
NICK WONG: Yeah, exactly.
And if we wanted to verify that--
what this is, it's only 473 characters.
COLTON OGDEN: So we're reading like 3,700 bytes of nonsense?
NICK WONG: Yeah.
Exactly.
It's just ridiculous amounts of nonsense.
COLTON OGDEN: [INAUDIBLE] in different memory.
[? KyLash, ?] thank you very much for following.
NICK WONG: So now, we remake this with a slightly smaller buffer size,
you'll notice that our gibberish gets smaller.
So our hypothesis is probably correct.
So then what we're going to want to do is
deal with how we actually read things into that buffer.
And this is where we're going to want to create a little bit
smarter read buffer.
So let's demoread--
COLTON OGDEN: Are we going to check for end of file?
Is that going to be part of the solution?
NICK WONG: Yeah, pretty much.
And basically, what we'll end up doing is just kind of initializing
the entire buffer to nothing, to 0.
COLTON OGDEN: Oh, and then just read in only until end of file--
NICK WONG: Yeah, exactly.
COLTON OGDEN: --and then it'll get the null string, and then that means--
[CLICKING TONGUE]-- we're done.
NICK WONG: Yep.
And then we're good.
Now, if we want to create a really smart read and write,
then maybe something that we would do is create a customs data structure in C
that keeps track of maybe which files descriptor it's attached to,
its own buffer, and which position it's at in the buffer.
And actually, you'll find that you require
roughly two or three of those things in order to get that to work.
So basically, what ends up happening is you
need to know where your imaginary buffer is, in its beginning and its end.
And what I mean by that, it's actually a little strange,
you want to know where it is, like what its positions would be in the file.
And so you can actually think about it as like, here's
your file, if I divide it up into segments by buffer size, then
it makes a lot of sense for me, at any time that I'm reading from the file,
I read in that entire buffer segment, is what we would probably want to do.
COLTON OGDEN: Right.
NICK WONG: Now, demoread is actually quite simple.
We don't have to do too much with it.
But let's say int demoread, and we're going to read from some file--
int file descriptor.
We're going to want to read into some char star buffer,
and we want to read in some number of bytes.
COLTON OGDEN: [? TheDuckThough ?] is saying,
"Let's go lower and [? writekernel ?] driver.
NICK WONG: I would argue I don't know how to do that, at the moment.
It's very possible that one day I will.
Very unlikely.
COLTON OGDEN: Are you taking CS161, or did you take it already?
NICK WONG: Yeah, I will be taking that next semester.
They might actually talk about that.
COLTON OGDEN: Yeah, I wonder if you guys do that kind of stuff in there.
NICK WONG: Which I think is very cool.
COLTON OGDEN: CS161 is the operating system class here at Harvard.
NICK WONG: Yes.
It's a class that I have waited for quite some time to be able to take.
COLTON OGDEN: Your whole life.
NICK WONG: Yeah, I've been very excited about taking that.
And so what we want demoread to do is actually only read into the buffer
basically whatever it could originally read.
So what we might do is some sort of memset of buffer 0--
we'll actually do sizeof buffer, to be consistent.
And then what we're going to try and do is read up to n bytes into that.
So int r equals read of file descriptor buffer n.
And you'll notice this keeps the same usage.
And then we might return something, r, here.
If I wanted to be a little bit more concise, then I could return read.
COLTON OGDEN: Oh, you forgot int in front of n.
NICK WONG: Oh, thank you.
Much appreciated.
COLTON OGDEN: Kudos to [? TheDuckThough ?]
for pointing that out in the chat.
NICK WONG: Nice.
COLTON OGDEN: People are on top of it.
NICK WONG: Yeah.
People watching the stream are always very on top of it.
It's very cool.
COLTON OGDEN: Thank goodness, because otherwise, I know for myself,
I miss so much.
NICK WONG: Yeah.
I miss all sorts of things frequently.
Oh, memset is a-- is that really a string?
I want to argue that there is a syscall--
oh, no there isn't.
That doesn't make any sense.
If you're writing in memory, there is no need for a syscall.
Of course.
And what I just mumbled to myself is that I was like,
oh, there should be a syscall, memcopy or memset or something.
They should be syscalls.
But of course, that doesn't make any sense.
[? syscalls ?] deal with going into the kernel space.
And in order to write to RAM, I don't actually
need to touch the kernel, which is why it's so fast.
So that would have made very little sense,
and I'm really glad I noticed that.
Cool.
So we now are noticing that we still have this problem of gibberish
at the very end.
And that's something that's totally reasonable for us to expect,
in that we did do actually a memset of size of the buffer into 0, blobdeeblah.
However, something that we might want to do is something more like this.
So what we will do is we can capture this output from demoread and use that.
And so what we can say is something like this.
while true-- and we'll break out of this using break.
while true int r equals demoread of standard--
oh, wait.
No, we're using file descriptor.
Buffer size and size of buffer.
Now, this return from demoread is the same
as the return from read, which is how many
bytes we actually ended up reading.
And so what we might do is something like demowrite of actually just r.
And from there, if r is less than or equal to 0,
then we break out of this loop.
COLTON OGDEN: You don't have access to true either, by the way,
because you're not including standard bool.
NICK WONG: Oh, awesome.
Got to love that.
COLTON OGDEN: Yeah, while 1.
NICK WONG: A lot of these sorts of things, I would argue, are not great.
And so it is our job to point them out to you.
This is an example of something that is not the best design.
COLTON OGDEN: You'd see it everywhere in like the '90s and '80s, though.
NICK WONG: I know, and that's the part that astonishes me,
is that that worked.
And you'll notice that this now does exactly what we would think,
where it actually cats the right thing out.
If we wanted to verify that test.txt to test2.txt, we can go ahead and do that.
This will take a little while, probably up to eight seconds or so.
COLTON OGDEN: [? Davins, ?] "Lmao, what are booleans in 2019?"
NICK WONG: No need for booleans.
You just use 1 and 0.
Or if you're even crazier, maybe you can find a way to use like empty string,
or like an empty list and--
I'm trying to think of something creative--
and like the number 42 as true.
You could get into some pretty weird things.
COLTON OGDEN: So what was it?
So r is going to be--
so you were breaking if r is less than or equal to 0.
So in which case--
NICK WONG: Right.
COLTON OGDEN: --is that going to be the case?
NICK WONG: So that will occur any time that there is an error in demoread,
or when we get to end of file.
COLTON OGDEN: OK.
NICK WONG: And so both of those mean that we break out of this while loop
and then close our file descriptor, and exit.
COLTON OGDEN: And then demoread is not going
to read the full 4,096 bytes if it's at the end and it goes beyond 4,086?
NICK WONG: Right.
It'll read up to 4,086 bytes.
COLTON OGDEN: And that's in demoread.
Can we see which part of that will be in demoread?
NICK WONG: Exactly.
So if we go into demoread, we are setting our entire buffer
to 0, which is actually not necessary, at the moment,
but I had it there originally.
And what this does is we actually do some sort of int r
equals the read of our file descriptor that we pass it into the buffer
up to n bytes.
And that's what we looking for.
And it returns up to n, or negative 1, if there was an error.
COLTON OGDEN: OK.
And then so the actual function call of it, we're reading this-- oh, 1,024.
OK.
NICK WONG: Right.
COLTON OGDEN: OK, I got it.
NICK WONG: Yeah.
Exactly.
And so you'll notice that that program to cat everything into test2.txt
took quite some time.
And so we're going to do strace one more time
and see what happened, because that is a little bit strange.
Excuse me.
And if you are looking at these and you're saying, oh, well, obviously this
would take a very long time, you'd be very right.
It is quite obvious, if you can remember,
if you scroll up to demowrite, we write everything one byte at a time.
And that is where you actually see this get quite slow.
Now, if your recall, our original program
that read and wrote everything one byte at a time,
that was eight seconds or so.
This one does not seem to be only eight seconds.
It takes quite some time.
COLTON OGDEN: [INAUDIBLE].
NICK WONG: And we're going to kind of--
well, strace.out is going to be quite large, in the sense that we are
reading and writing 13 million lines.
Now, you'll notice we did this read of 1,024.
That was great.
And then we proceeded to write 1,024 times, which is a huge pain.
So if we wanted to speed things up a little bit,
then we could still increase this buffer size of ours to 4,096.
And we can do this.
Oh, wait.
I have to remake.
We go through, and it will get slightly faster.
Now, strace is going-- oh, I shouldn't run that with strace.
What I actually wanted to run was this, to kind of point out
to us just how much faster it got.
Now, this should be some weird hybrid of eight seconds and 0.0015 seconds.
Now, we might end up with something a little bit different from that.
We could hypothesize as to why that's true.
And then the next step would obviously be
to write a write buffer, which is a little bit different,
in that whenever you go to close the file,
you also need to flush the buffer to the file.
And that would require a little bit more work on our part.
I guess I could have actually just had that be our goal the whole way,
but we have now, at this point in time, talked
about how to build a version of cat using the standard in and out,
standard I/O.
And that's kind of cool.
It doesn't have all the features and functionality of cat, but it does work.
And then we went and moved to something that
is a little bit of a hybrid between low-level syscalls and standard I/O.
And then we went all the way into a syscall version of cat
that has some of its features, in that it can read pretty quickly,
it has a buffered read, but its writing is pretty terrible.
And so that is something that you might be interested in exploring a little bit
more.
It's very much a worthwhile exercise to see
if you can implement different kinds of buffer,
as opposed to the one we're doing here, which would roughly
translate to a single-slot buffer.
We have a single slot that we are using to read into and then write out of.
However, we could very easily build a multi-slot buffer,
where I read into one slot, and then into another, maybe up to four or more.
And maybe have some reason and policy by which we kick something out of a slot
and fill it with a new set of data.
There's all sorts of things you can do with that,
and I would certainly recommend trying out all sorts of things
with how you actually deal with how your computer runs.
I think it's very cool.
So that concludes all that I have, as far as the stream goes, actually.
COLTON OGDEN: We are at the 2 hour and 10 minute mark.
So thanks, everybody who came in today to check out the stream.
I'm going to take us to the wide shot.
Actually, no, screensaver.
We've got to do a screensaver.
That's the--
NICK WONG: That is very important.
COLTON OGDEN: We've taken it out with that every time.
We'll leave it on this for just a minute,
while we take some last questions.
So apologies if we didn't get everybody's questions today.
NICK WONG: Yes.
COLTON OGDEN: There was a lot, which is awesome.
I really appreciate everybody's involvement today, everybody tuning in.
And a lot of people tuned in really early too--
NICK WONG: Yeah, that was awesome.
COLTON OGDEN: --which was super awesome.
Join us for the next one.
Actually, I'm curious, it's the end of the semester coming up pretty soon.
This might actually be the last stream until 2019--
NICK WONG: Wow.
COLTON OGDEN: --which is kind of ridiculous.
NICK WONG: That's crazy.
COLTON OGDEN: Yeah, we have the fair this week,
and then next week everybody's leaving.
I'm leaving town, actually.
I'm going to California.
NICK WONG: Hey.
COLTON OGDEN: You're from California--
NICK WONG: Oh, sweet.
COLTON OGDEN: --so you'll be going to California.
NICK WONG: I'll be going to California, too.
COLTON OGDEN: Nick and I are going to California.
I don't have a streaming setup over there,
so we might not have another stream until 2019.
But if that's the case, it'll be January 2nd or 3rd probably.
And then we'll be doing weekly streams, like multiple times per week,
going after that.
NICK WONG: Yeah.
COLTON OGDEN: You'll be part of that, I presume.
NICK WONG: No, I will certainly be there.
We have like Raspberry Pi is coming up.
COLTON OGDEN: Raspberry Pi.
NICK WONG: We'll deal with some hardware stuff, maybe even get a camera on one.
We'll have, I think, Kali Linux coming up.
COLTON OGDEN: Oh, the Kali Linux.
Yeah, the White Hat--
NICK WONG: Live CTF going on.
We will have--
I don't know.
Did David do a Docker one?
COLTON OGDEN: He's going to do a Docker one.
Yeah, he wants to do a Docker one.
NICK WONG: OK, then that one will be there.
We have all sorts of things going on.
Oh, Bitcoin we'll talk about.
COLTON OGDEN: Oh, yes.
True.
NICK WONG: We'll talk about Bitcoin.
And I guess we'll lump Blockchain into that, just because that's also
very interesting.
Maybe you like an API build, like build a RESTful API.
COLTON OGDEN: Oh, a RESTful API would be super cool.
NICK WONG: That'd be kind of cool.
COLTON OGDEN: Yeah, get the web devs in there.
NICK WONG: All of our web devs will be happy.
We might even build a Swift app.
COLTON OGDEN: Oh, a Swift app would be cool.
NICK WONG: I think that'd be really cool.
I had totally forgot about that until the [? CSVD ?] Hackathon.
Then everyone was like, oh, Swift, I love Swift.
And I don't really like Swift that much, but I do love building apps in it,
and I think that's really cool.
So I think that'd pretty fun.
COLTON OGDEN: Yeah.
Yeah, all of the things.
NICK WONG: Yeah.
We'll go into all sorts of things.
COLTON OGDEN: Yeah.
We have a lot coming in, and then I have--
we'll be doing probably once or twice a week gaming-related--
NICK WONG: Sweet.
COLTON OGDEN: --programming tutorials and streams.
So tune in for those, starting around January 2nd or 3rd.
And I'll get you to figure out what the first day will be.
We'll be on the Facebook group, so definitely message us.
If we don't see you or hear from you until then,
well, definitely have a great holiday yourself--
NICK WONG: Yeah.
COLTON OGDEN: --wherever you are in the world,
and whatever holiday you might be celebrating--
NICK WONG: People are all over the place.
It's awesome.
COLTON OGDEN: Yeah, we've got people all over the world, United States
and then abroad.
We have India, we have Italy, Greece, and all sorts of places.
New Zealand.
[? Brenda ?] repping New Zealand is a big one.
NICK WONG: Literally all over the world.
COLTON OGDEN: Belgium, Netherlands, all kinds of places.
Lots of great message in the chat saying--
NICK WONG: Yeah, thank you guys so much.
COLTON OGDEN: Happy holidays.
Yeah, same to all of you.
[? Bhavic Knight, ?] [? MeowCat, ?] [? 42Force, ?] [? Erenay, ?]
[? 42Anto42, ?] [? UnsignedEd, ?] [? ShaneHughes1972, ?] California--
NICK WONG: California.
COLTON OGDEN: It says California.
NICK WONG: Me too, yeah.
COLTON OGDEN: California's--
NICK WONG: California.
COLTON OGDEN: --good stuff.
California's good stuff.
And then [? AzayAndre, ?] [? Fatma, ?] yeah.
NICK WONG: Yeah.
Thank you guys very much.
COLTON OGDEN: It's going to be--
NICK WONG: Oh, shoot.
I forgot that someone in the chat said, what did you do in binary exploitation?
And I totally forgot to talk about binary functions and things.
We'll mention that.
We'll do a whole stream on it maybe.
COLTON OGDEN: We could do a part two.
We'll do a part two maybe on that.
There's just so--
NICK WONG: There's so many things to do.
COLTON OGDEN: --much to cover, and two hours
is not a lot of time for all of that.
NICK WONG: No, it really is not.
COLTON OGDEN: But yeah.
Thanks, everybody again, so much.
This has been CS50 on Twitch with Nick Wong.
My name is Colton Ogden, and tune in next time.
Tune in in January, if we don't do another scream until then.
And have an awesome holiday season.
NICK WONG: Yeah.
COLTON OGDEN: All right.
NICK WONG: Enjoy.
See you guys.
COLTON OGDEN: Happy holidays from CS50.