SICP programming study group

[Synasta]

Anybody interested in learning some programming? I'll be studying the fundamentals using this textbook and free video lectures provided. It's a very highly recommended textbook, although a caveat is that you may not be getting the instant gratification that keep most self taught programmers going e.g. seeing the benefits of building up your own website bit by bit. Hit me up with a PM if you're interested in forming a tag buddy team to motivate each other or if more are interested a study group

 

[Synasta]

I did a lil reading, sorry this isn't so much a book on programming, it's more focused on the theoretical parts of Computer Science, and even then, the creators of this course/textbook claims that the term itself is a misnomer (i'll get into more of that later). It is more on getting you to think on why one design approach is better than another when it comes to implementing algorithms and programs. I can't really say much about that since i'm still stuck on chapter 1. But here's a short summary for those interested but not biting yet.

The Black Box Abstraction:
As in most engineering disciplines, we use this technique in CS to help in managing systems of increasing complexity. For example, we have a program for calculating the square root of a number. Within that program, we have subsequent programs for refining the guess of the square root, how to tell whether the guess of the square root is within an acceptable tolerance, etc. But these details are unimportant. They are kept out of sight, hence the label "Black Box". By decomposing functions in such a manner, we minimize unnecessary details when looking at the big picture of how the system works.

The limits of programming:
Take civil and electrical engineering for instance. When constructing a building or designing an electrical circuit, we have to consider stresses, loads, noise (signal distortion), etc. Hence our design is limited by the natural laws of physics and we have to work within these constraints, however in building large software systems it is very unlikely we will hit a physical limit by asking too much from the computer in terms of performance (at least when we're beginners)

Declarative and imperative knowledge:
Computer science is interested in imperative knowledge which is how-to knowledge, while declarative knowledge is what-is knowledge (i.e. mathematics) Going back to our square root example, if you had the formal definition of what is a square root, it wouldn't tell you anything about how to compute a square root from any number. That is where imperative knowledge comes into play, it is interested with describing procedures, processes and specific steps taken in order to find out new information.

 

[AnimeNewbie]

You typed so much but neglected to mention the computer language used. Anyone who is interested in scheme or/and functional language?

 

[Synasta]

Sorry you're right my bad, i forgot to mention that. Here's an excerpt from the book:

If Lisp is not a mainstream language, why are we using it as the framework for our discussion of programming? Because the language possesses unique features that make it an excellent medium for studying important programming constructs and data structures and for relating them to the linguistic features that support them. The most significant of these features is the fact that Lisp descriptions of processes, called procedures, can themselves be represented and manipulated as Lisp data. The importance of this is that there are powerful program-design techniques that rely on the ability to blur the traditional distinction between ``passive'' data and ``active'' processes. As we shall discover, Lisp's flexibility in handling procedures as data makes it one of the most convenient languages in existence for exploring these techniques. The ability to represent procedures as data also makes Lisp an excellent language for writing programs that must manipulate other programs as data, such as the interpreters and compilers that support computer languages. Above and beyond these considerations, programming in Lisp is great fun.

 

[davidktw]

Sorry you're right my bad, i forgot to mention that. Here's an excerpt from the book:

If Lisp is not a mainstream language, why are we using it as the framework for our discussion of programming? Because the language possesses unique features that make it an excellent medium for studying important programming constructs and data structures and for relating them to the linguistic features that support them. The most significant of these features is the fact that Lisp descriptions of processes, called procedures, can themselves be represented and manipulated as Lisp data. The importance of this is that there are powerful program-design techniques that rely on the ability to blur the traditional distinction between ``passive'' data and ``active'' processes. As we shall discover, Lisp's flexibility in handling procedures as data makes it one of the most convenient languages in existence for exploring these techniques. The ability to represent procedures as data also makes Lisp an excellent language for writing programs that must manipulate other programs as data, such as the interpreters and compilers that support computer languages. Above and beyond these considerations, programming in Lisp is great fun.

First class functions are found a few other languages such as JavaScript and Perl too. If you are interested in programming languages design and its concepts, perhaps NUS has a few courses u might be interested, including compiler design course which I have attended

 

[Synasta]

Thanks for the suggestions but not really suited for me (long story) self study for me now the book is really hard though. I'm still not past chapter 1. I have more summaries of new concepts but i think i will save for them another time when i have compressed them more succinctly.

 

[davidktw]

Thanks for the suggestions but not really suited for me (long story) self study for me now the book is really hard though. I'm still not past chapter 1. I have more summaries of new concepts but i think i will save for them another time when i have compressed them more succinctly.

I think you can revisit this when you feel you have understood more concepts that this book introduced. Somehow I don't think this book is for starters.

The topics introduced no doubt are general generic concepts, but in my opinion, it takes a well learnt developer to truly appreciate the generic concepts. Things like huffman codes, lambda calculus, closures, registers computing, metalinguistics are in no way under the syllabus for most beginners.

However, should you already is an intermediate to advance developer with a couple years of computing knowledge and also a couple of years of industrial experiences plus interested in the theoretical concepts of computer science, then I must say these courses are especially enlightening to your knowledge.

Until you really understood the essences of modular development, going through numerous industrial debugging and source code reading of others works, black or white boxes probably just are boxes with little meaning. I know anyone can read the differences between white and black boxes testing, but how about applying it across solutions and architectures ? I believe only when you are doing it across different environments and integrating across the systems where you will find how enlightening it is.

Hence my opinion is for starters, go get your hands dirty instead. Learn the hard way, understand the smart ways, exercise the smart ways and revisit these syllabus, you will gain more where you can correlate with what you have came across or did. That's where you see what more can be done and what you have not thought about.

 

[Synasta]

I'm not really sure about the specific examples you cited (sorry, wasted on me ) but i get the general gist of what you are saying. But being a stickler for efficient learning, i remember reading something that convinced me to go use this book as a self study guide; programming should be taught as an approach to problem solving and as structured logical thinking. Once a person can understand fundamentals and the core concepts, the individual languages don't matter all that much; and computer science is what differentiates a programmer and a code monkey.

So, "suffer" now and get a good understanding of the fundamentals, rather than dive into programming right away and possibly learn bad programming habits meanwhile. I could go into a long spiel about the process of learning efficiently from pop psychology books, but that's a story for another day

Not sure what you mean by getting my hands dirty anyway? At this point, my knowledge of lisp is nearly non existent for me to create any worthwhile program, and i am still able to understand the book and lectures somewhat even if its at a glacial pace. Any suggestions (but i am loath to give up now, i promised myself at the start of the semester i would get this done one way or another, and my exams are just over )

 

[davidktw]

I'm not really sure about the specific examples you cited (sorry, wasted on me ) but i get the general gist of what you are saying. But being a stickler for efficient learning, i remember reading something that convinced me to go use this book as a self study guide; programming should be taught as an approach to problem solving and as structured logical thinking. Once a person can understand fundamentals and the core concepts, the individual languages don't matter all that much; and computer science is what differentiates a programmer and a code monkey.

So, "suffer" now and get a good understanding of the fundamentals, rather than dive into programming right away and possibly learn bad programming habits meanwhile. I could go into a long spiel about the process of learning efficiently from pop psychology books, but that's a story for another day

Not sure what you mean by getting my hands dirty anyway? At this point, my knowledge of lisp is nearly non existent for me to create any worthwhile program, and i am still able to understand the book and lectures somewhat even if its at a glacial pace. Any suggestions (but i am loath to give up now, i promised myself at the start of the semester i would get this done one way or another, and my exams are just over )

My words are for your consideration. you can decide if you want to plough through it. To be frank, even at my level, such topics are not for the faint of heart. I have been learning programming since secondary and if you want to start counting my official programming education, it's since JC1 which is more than 12years of computer science knowledge. I pride myself always learning new technologies and going for hardcore computer science to challenge myself. Yet when I look at the topics discussed, they requires quite a knowledge on different computing discipline to full appreciate those concepts that are presented.

Just to list a couple.
1) Modular development and testing - This topic has been taught in early computer science education, yet most students often find confused about their usefulness and how to apply them properly. I still remember the younger days where I find unit testing is still for the obvious and seems pointless. However, only when you start working with a team and you witness how one person negligence can undermine the effort of the whole team, where you start to appreciate that obvious are normally taken granted for. It's through these black and white box testing of the most obvious assertions that builds the code confidence in a project. This topic alone takes a lifetime, if you ask me, to appreciate. Each programming methodology comes with a different testing framework and each project team may requires something slightly different.

Hence I don't except starters to fully grasp the fundamentals of such framework. They only understand it when they hit the wall. Smart people hit the wall once and learn to work around it with frameworks. Dull ones keep on hitting it non-stop, probably work through it with some help, but never learn how to avoid it the next time.

2) Functional topics are normally not for beginners because they demand a radical thinking which is less straight forward, but intuitive, if you ask me. Lambda calculus form the basis of functional methodology. However, it takes a good course to impart the idea about functional evaluation. I took a 3rd year course in Programming Languages which discuss about functional languages, how they relate to imperative programming, and later also got myself into a Compiler design course which discuss more stuffs like JVM and even implemented a variant of Lambda language using Java. Without a strong foundation in imperative programming, the idea of how evaluation and side-effects are hard to grasp. You also need to get yourself a good understanding of discrete mathematics with good algorithm programming courses to fully appreciate the recursive nature of functional programming. Skillsets like traversing a tree, building a linked list recursively, doing quicksort recursively, and also dynamic programming and unwrapping a recursive function into a iterative stack function will helps alot in modelling recursive functions.

3) Scoping is extremely interesting but mind bogging too. Static scoping, lexical scoping, dynamic scoping and when u have OOP, there are others too, are something u can't just read up. You have to experience it coding real applications using different programming languages such as Java, Perl, PHP, and some real functional languages. Each language construct have slightly different semantics though they bear on the same scoping framework. Especially for closures and their effects, something that is kinda a hype in recent imperative languages advancement to more functional behaviour, can be helpful and also a good source of confusion. If you haven't met them, you won't really understand them. Hence for some learning, I believe "getting your hands dirty", meaning you do it, hit some walls, understand the behaviour, then read about them in the context and how to solve them, makes better learning than just theory. Theory can often reinforce your understanding, expanding the context into generic ideas and concepts, but doesn't have to be the starting point all the time.

Enough said for all. For one thing, I'm not discouraging you from learning from such course. But it seems you are getting into something that you have lack in experiences and breadth knowledge to absorb. I learn about Huffman coding from compression algorithms and JPEG implementation, something out of most programming tasks because we often use it and don't really want to understand the underlying algorithms for the task at hand is of higher complexity. With such knowledge, you will appreciate this course more. Hence I urge you to give it a miss and revisit it when u feel you are ready. Download it if you must so that you can read it again when you have more in you to grasp the book.

Cheers.

 

[Synasta]

Hi, i am really appreciative of the fact that you typed all these for my consideration. What route would you recommend then?

 

[davidktw]

Hi, i am really appreciative of the fact that you typed all these for my consideration. What route would you recommend then?

Are you a student now or already working ? If you are a student, will you be considering pursuing a computer science degree ? That is a proper recommendation that I will encourage.

 

[Synasta]

Studying engineering as my major already, it's my holiday now. I want to study computer science on my own time anyway.

 

[davidktw]

Studying engineering as my major already, it's my holiday now. I want to study computer science on my own time anyway.

To be frank, there are vast disciplines in computer science than just daily talking about programming languages and methodologies. Since you are already in the engineer faculty, then I suppose you can consider taking up some Computer Science cross faculties modules if they are not in your existing core curriculum.

There should be modules of the following nature
1) Computer Architectures
2) Data Structures and Algorithms
3) Understanding Programming Languages
4) Compiler Design
5) Operating Systems Design
6) Networking
7) Database

Since you have set a pretty high expectation to understand core Computer Science, then I see not excuse to not know any of the above subjects. If you want to be able to read pass the SICP, then you will require prior knowledge to some of the subjects mentioned in order to appreciate how you can perform them in Lisp, or any functional languages.

For example, as I skim thorough the chapters, I saw a part that talk about trees construction and traversal. You will need knowledge from a "Data Structures and Algorithm" course. It will teach you about the mathematical premises about trees. It will teach you about what trees structures use cases are, the different implementation of trees, the functions and properties to manage a tree, the time & space complexities of trees. If you can't understand this, you wouldn't be able to appreciate the beauty of functional languages operating on such recursive data structures.

Simply to say, when you try to read up the SICP without these prior knowledge, you are basically jumping the gun. I can almost be sure even after visiting the course once, you will likely just learn about the syntax of lisp, but not the essence of the course which is what you are after - Grand understanding of Computer Science and its applications.

Without prior knowledge about register based and stack based processors design, which is taught in (1), you will not be able to appreciate what are "registers" in the computer design and what mathematical challenges are involved in register allocation found in programming languages.

I'm not trying to scare you, but your expectation is huge and hence the cost involved in learning is huge. All of the above courses normally fall under the core curriculum of Computer Science courses. In my opinion, it is the only enlightenment part, if you truly wants to end up as a versatile computer scientist whom can venture in grand methodologies and computer science applications.

 

[Synasta]

My major has fixed modules that i can take, and no flexibility to take cross-disciplinary ones - i study at the new university, SIT. So yup, formal learning is no good. Furthermore, i don't rely very much on lectures in the modules i already take because i'm hearing impaired, no assistance. So i'm used to self sufficiency in studying. So is there any alternative textbook/online course you would recommend? I already actually have some familiarity with lisp having read Land of Lisp (very cheery way to learn ) before, so i can understand the examples somewhat.

I also saw the online course based on this textbook was an introduction to CS for undergrads. Surely it can't be that bad. And i think when i say computer science, i don't mean to be that far reaching or ambitious. Just a vague idea that motivates me of being able to program better, sorry :S

 

[davidktw]

My major has fixed modules that i can take, and no flexibility to take cross-disciplinary ones - i study at the new university, SIT. So yup, formal learning is no good. Furthermore, i don't rely very much on lectures in the modules i already take because i'm hearing impaired, no assistance. So i'm used to self sufficiency in studying. So is there any alternative textbook/online course you would recommend? I already actually have some familiarity with lisp having read Land of Lisp (very cheery way to learn ) before, so i can understand the examples somewhat.

I also saw the online course based on this textbook was an introduction to CS for undergrads. Surely it can't be that bad. And i think when i say computer science, i don't mean to be that far reaching or ambitious. Just a vague idea that motivates me of being able to program better, sorry :S

Forgive me for saying so, that curriculum is written in 1993 which is nearly 20years from now. Back in those days, people are like Linus, when they don't have an OS, they write one. When they don't have a word processor, they wrote one. Now .... Times changed, when you don't have a OS, you BUY ONE. Generally the syllabus written in SICP, isn't irrelevant, but you will find less students todays have the capability to absorb. Am I being pessimistic ? I think I have seen quite a number of students and colleagues and others doing IT subpar. I'm not trying to be elite here, but unfortunately, this is reality we need to accept. I will be surprised if there are still students on their own implement RED-BLACK trees instead of using what is already found in JDK or in other C++ implementation. How about who, whom is reading this post knows that the Java implementation of sorting in one of the classes method uses both quicksort and another simpler sort algorithm in a single method ? You have to read the Java Software Development Kit's source code to know when the array input size exceed a threshold, it turns to quick sort, if not, quicksort is slower than an insertion sort, if I can still remember. This is the kind of optimization Sun's developer did behind the interfaces.

Lisp is nice, I learnt CLisp when doing my Knowledge Based course. Definitely need a good editor if you don't want to get lost in the excessive brackets in the syntax. How here I'm not talking about Lisp or any other languages written in the course. It's about the topics that are written. I don't think modern computing books that you can find nowadays uses such advance scenarios to example programming. Those are really advance topics, NOT for modern starters.

If you are keen in computing, okay, get a developer job with an SME. I can assure you can learn alot through dirty works. If you want to get into the limelight, you need a skillset to get you in first. Try learning Java or .NET. Lisp will not get you in, unless you are into research. I seriously don't find any job description in Singapore that requires Lisp. If you can't get such a job, what you learnt probably are what I will call "Kid's Play" because you will not have really industrial requirements that require you to be innovative. The advancement of human technologies are normally driven by needs. I personally find it hard to set your own milestone, they tend to be unrealistic with little commercial values which makes requirements REALISTIC

It's just me and what I go through to share, getting yourself into the main track is important. That's where you will come across opportunities for you to advance further. Enterprise world is both a bless and curse, but you get to choose how you sail through it.

I recommend you pick up a modern Java/.NET book, read it up. Also look at the curriculum in your school offering Computer Science, get a book to read up on other subjects that I have mentioned in the earlier post.

Since you are in the engineering, I believe this book will bring you more awareness into Computer Design. "Computer Organization and Design" @ Computer Organization and Design, Fourth Edition: The Hardware/Software Interface (The Morgan Kaufmann Series in Computer Architecture and Design): David A. Patterson, John L. Hennessy: 9780123747501: Amazon.com: Books

Then try reading "Introduction to Algorithms" @ Introduction to Algorithms, Second Edition - Books | Algorithms and Data Structures. This will build up good data structures and algorithms foundation in every scientists, not just computing. It will be extremely helpful in your engineering skill sets too. It helps to discover problem modelling and train your thoughts in solving problem.

I can't say more about discrete mathematics, but it is the core foundation to logics and reasonings. However, this will require you to attend a course, it's not something easy to pick up from books.

 

[Synasta]

Isn't Java, .NET, etc.for web development? I was hoping more to apply programming knowledge to engineering domains for example finite element analysis, the market for web development seems oversaturated for me. I graduate in 2 years time, being a very accelerated course with holidays being too short for me to get a stint as an intern at any SME and so i was hoping to just gratify my intellectual curiosity by exploring more theoretical avenues and keep my interest level/motivation high instead of making myself learn "practical" stuff like Java and .NET.

Anyhow, i checked out both of your book recommendations and Introduction to Algorithms looked more interesting of the two, i will take a look, thanks SICP textbook might have been abit more than i could handle, all the exercises are extremely difficult and require some mathematical background, though the video lectures seem easier to follow.

 

[davidktw]

Isn't Java, .NET, etc.for web development? I was hoping more to apply programming knowledge to engineering domains for example finite element analysis, the market for web development seems oversaturated for me. I graduate in 2 years time, being a very accelerated course with holidays being too short for me to get a stint as an intern at any SME and so i was hoping to just gratify my intellectual curiosity by exploring more theoretical avenues and keep my interest level/motivation high instead of making myself learn "practical" stuff like Java and .NET.

Anyhow, i checked out both of your book recommendations and Introduction to Algorithms looked more interesting of the two, i will take a look, thanks SICP textbook might have been abit more than i could handle, all the exercises are extremely difficult and require some mathematical background, though the video lectures seem easier to follow.

First I would like to point out your misconception about programming languages. There is no such thing as Web development programming language. No doubt Java is brought into light in years where the Web is striving, however, its predecessor know as Oak, is design ground up for embedded system and hence very much related to your engineering purpose. As much as C or C++ are used extensively in engineering projects, no one is stopping one from using Python, Perl, Ruby, Java, Javascript for any kind of computing feat. It's all limited by your thoughts.

All these programming languages passed the turing test and hence they are just good enough for any development work of any nature. What set aside certain programming language as being fore web development are the following reasons:

1) Perception, because one sees certain languages more popular and prominently used in certain use cases, they tend to be biased and affix a purpose to them.

2) Libraries, projects and modules. Some programming languages with Enterprise and open community support get to have alot of libraries written for certain use cases. These libraries make the programming language easier to accomplish certain task and hence makes more people using it. The cause and effect is mutual.

No one stop you from using Java to write games, robotics programming like LeJOS in Lego mindstorm. No one is stopping you from developing genetics programming for solving the human genome using Java, nor is stopping you from running a JVM in the airplane or navigating a rocket in NASA. For as far as I am aware, Java has stretches to so many industries and you will not be surprise to find libraries doing all sorts of industrial stuffs developed.

Look at Python, though it is also a web development language (as you called it), it is used in stock market analysis in some works uding SciPy and Numpy library. Ruby is used in Google sketchup as a plugin library to perform 3D works. I just wrote one recently for fun and also for my friend.

What is so practical about web development ? There are so much science and mathematical concepts applied in the very core platform globally used today. For example security frameworks and algorithms. Where do you think its more utilized ? Banks ? Government ? Or the commercial industry ? I can almost be sure you will find much of today's security frameworks are built to such robustness because of the Internet. How about network ? Where do you think today's network core is found ? Telecommunication or the Internet ? Now we have cloud computing where resources are cheaply available for anyone to start large scale architecture for a few hours of computing, where do you think the skill set of large scale design grows ? Large scale like hundreds to thousands of servers ? Don't underestimate the complexity of web development and the methodologies that grows out of the needs in it. Everyday humans are challenged in the Internet from each other across the globe. I can quote you more theory than you can anticipate just from the Internet alone. You will find algorithms and data structures fully exercised in the web development arena. We are dealing with so much data every day that database methodologies changes form year to year. First we have high performance database, then comes clustered database and now multi master databases that slack down on consistency to achieve better efficient and more scalable design.

If you are interested in functional aspect of programming languages, you will find new languages such as Scala, a programming language running ontop of Java virtual machine, but meant for functional design. Some companies have tried on Scala, finding a fit for in in the projects they do.

What I would like to introduce to you is not what programming languages you chose or what you do. It's how you think, and how you keep your mind open. Only with an open-mind and always challenging yourself intellectually, learning new stuffs whether it is obvious or complicated, that makes you learn as much as possible.

If I were like you thinking practical is not the way to go, why do you think now I can discuss with you such vast knowledge into different areas of computing, ranging from programming languages, system design, network paradigm, security and cloud computing ? Give a good thought about it and see if I'm misleading you

 

[Synasta]

Thanks for clearing up my misconceptions. You're right, i was completely misunderstanding things. Although, i'm not too sure what you mean by these programming languages having passed the turing test. Isn't it a test to check whether a computer is intelligent enough to simulate human behavior? Also, on another hand, why isn't lisp actively utilized in web development, being one of the older languages around?

 

[davidktw]

Thanks for clearing up my misconceptions. You're right, i was completely misunderstanding things. Although, i'm not too sure what you mean by these programming languages having passed the turing test. Isn't it a test to check whether a computer is intelligent enough to simulate human behavior? Also, on another hand, why isn't lisp actively utilized in web development, being one of the older languages around?

You are right about my mistake, it should be Turing Completeness that I should be using, not Turing test. What am I typing....

Well just the brackets alone will make your head dizzy, that's one good reason you wouldn't be using to code most web development works. Especially when you see how god damn convoluted in nowadays people code.

Next is somehow it takes good training for a developer to think "recursively". And how about writing mathematical expression in postfix notation instead of infix ? I have no issue reading postfix, because I'm pretty used to it, But if you want a starter or a subpar developer to read (/ (+ 1 2) (sqrt 4)), I'm sure you will find very little developers able to do it.

They will certainly find (1 + 2) / sqrt(4) alot more easier to grasps.

For obvious reason of top-down approach, developers are also more accustom to imperative paradigm than functional paradigm.

They need to start thinking evaluation more than side effect.

For eg:

a = 1 + 2

The evaluation is 1 + 2. The side effect is "a". Means if I were to write functionally, my concern would be (1 + 2) where "1" and "2" are the input and the output is something I want to pass on to somewhere. I'm not too concern with assignment to "a" and then work on something else.

Look here

a = 1 + 2;
doSomething(a);

versus

(doSomething (+ 1 2))

The first imperative programming is systematic, top-down approach. Most developers tend to, or at least feels comfortable to first settle "1 + 2". Store somewhere, then the brain think about what to do next, like writing an essay.

But if you are writing in functional approach, you either thing thorough it that you already want to doSomething about the input, or you first write

(+ 1 2)

Then you think you want to do something, hence wrap doSomething around it.

(doSomething (+ 1 2))

The mind thinks differently, at least thats how I perceive it. You need to be rather well trained in thoughts to do well functionally. Mind bogging to most if I must say so.

However while I am not aware, I probably think some people can think this functional manner, like I do, hence I love functional languages and their beauty. The concepts of evaluation, substitution, reduction and more are intriguing, but because you find practically there are not so much ready libraries in the industry, working with functional languages can lend yourself in a very tedious predicament.

After all, we are trying to get something done for the renumeration, not trying to experience something done. If you are keen in functional paradigm, start reading some online lectures on Lambda Calculus, it can be intriguing and also mind bogging. But it is the mathematical basis of functional programming and also imperative, I believe.

 

[Synasta]

I guess i am thankful my first real exposure to programming is through Land of Lisp, i can switch between both ways of reading expressions at a very basic level only though. What kind of stuff are you studying now, if study is an appropriate word?