So after yet another disappointing realization that I don't have the skills that most employers want, I have decided to forego brushing up on Clojure, or learning D...or OpenGL. Sigh.
Unfortunately, I work in a kind of half-way world where my skills don't seem to be valued. I am not a firmware developer or device driver developer (anymore), and I don't have enterprise skills that most other businesses want (my SQL knowledge is very basic and I know very little about web development either back end or front end).
So for the last 3 weeks or so, I have been learning OSGi. I have made an Eclipse plugin before, but that was mostly just trial and error because the documentation was so bad (even the SWT documentation was pretty bad, but I somehow managed to make a JFace based plugin). So this is the first time that I've really dived into OSGi, and I dived into it because of something I've been seeing at my workplace: non-modularity.
Spaghetti code has of course, a bad connotation to it. But I think some of its original meaning has been lost. Spaghetti code is code in which codepaths and/or dependencies get so interwoven that it is no longer feasible or possible to change one thing without that change cascading into a lot of other code. When you have non-modular code you almost by definition have spaghetti code. I have seen first hand now what happens when you only need some classes for a project, but because those classes have dependencies on other classes, and those classes on other classes (ad nauseum), you get into a monolithic all-or-nothing scenario.
I decided to learn OSGi precisely to combat this problem (despite the offending non-modular platform not being in Java). What I am discovering is that it is harder to design applications than it is to code them. Admittedly, I'm getting a bit stuck on Services at the moment, but to my mind, the hardest part is just structuring your application into modules in the first place. I am also hoping that when Java 8 comes out along with project Jigsaw, it won't be too much of a transition for me to use a more modular approach to programming. Also, I can finally (hopefully) figure out what Inversion of Control and Dependency Injection are. I've already been looking at iPOJO in felix, and it seems interesting, though a bit confusing. Component based frameworks have a lot in common with modularity, and they seem to work well together.
If there's one thing I've discovered being a SDET for the last 3.5 years, it's that there is definitely a need for engineers who can bridge the hardware/software/enterprise gap. It astounds me that the SCM team doesn't understand what "Software as a Service" is. Or what Service Oriented Architecture is. Hell, even just trying to show what distributed computing is was a challenge. I have had an opportunity now to see several organizations "Test Automation Frameworks", and to say that they were....to put it politely...behind the times is an understatement. State of the art to many test groups is to just have a pile of scripts written in a ton of languages with no means to tie together Requirements -> TestCases -> Scripts -> automated installation -> automatic argument passing. They just pass on by tribal knowledge that Script A is for TestCase B, and that you have to install a whole bunch of software on the system under test. Oh, and the tester will have to manually fill in all those annoying little things, like when they started the test, when it ended, and if it passed or failed (not to mention manually uploading log files or other useful debugging info).
Being an SDET in a hardware company is a bit challenging, because I have to not just provide an automation framework, but an ad-hoc one too. While the automation aspect mostly just deals with the enterprise side of computing, the ad-hoc tool creation requires understanding hardware, firmware, and drivers. If you talk to many Computer Science grads today, talk about Interrupts, Stack frames, logic analyzers, IOCTL's, SERDES or the finer points of memory allocation....and they will probably just gaze at you with blank eyes. The majority of CS majors I have met only took cursory C or C++ classes, and took neither Logic Design nor Microcontrollers. Conversely, Electrical Engineers don't understand the principles of good software engineering, and often have to be dragged into using Revision Control, given an explanation why writing 1000 line functions is not a good idea (inline them if you are worried about performance), that creating unit tests is helpful, and told that copying and pasting code is usually a really bad idea.
I definitely think there is a place for people like me. I'm not a device driver guru, but I have done it in the past. Ditto for embedded firmware (I used to write assembly a little even). I'm not a SQL master, but I can create simple tables with constraints. I don't understand load balancing for web servers, but I can write a simple servlet for Jetty to enable web services. But I think the most important skill I have of all is figuring out the technology that should be used. Maybe because I'm not a master at any one thing, but I know a broader swath of technologies than even many engineers with 20 years of experience who has been specializing, I know about many ways to tackle a problem.
Perhaps one day, an employer will discover that it's not so much about the skills you have now, it's about how quickly you can adapt and learn, and how well an engineer can integrate all of his skills and knowledge together.
Saturday, March 31, 2012
Sunday, February 19, 2012
Laziness in clojure. Or "Why didn't I see it was just induction?"
Ever have one of those "A-ha!" moments of clarity? One of those, "ohh, so that's what Eureka means" events of enlightenment?
So people who have been doing functional programming for awhile, and those more astute than I will probably go, "well, yeah...duh", but I just had a great realization that creating lazy sequences is really just the inverse of recursion. And what's the inverse of recursion? Induction.
So, I'm not going to go into what induction is exactly (whether it be mathematically speaking from a weak or strong perspective).
I decided to use a function I've written recursively several times that is of practical benefit (kinda) and turn it into a lazy sequence instead. Basically this is an interest calculator, given a certain starting principal, an amount that is saved (per year), and an interest. For my recursive solution, I also give a number of years, and in this case, I walk up to the number of years (starting with year 1). But for a lazy sequence, this isn't necessary. This is because, like induction, you don't try to hit the base case.
First, let's look at the recursive solution, then compare it to the lazy one.
I did add a little print here so you can see how the interest rate accumulates. But this is your basic tail-call recursive solution to a problem like this. You would call it like this:
(savings 0 4000 1.07 10)
Which would calculate how much you would save in 10 years with a starting principal of 0 dollars, saving 4000 a year, and at a 1.07% total interest (yeah, pretty good I admit) for a total of 10 years.
But what about a lazy sequence solution? And why would you want a lazy solution anyway? Let's look at the lazy solution first. In some ways, it is easier.
Notice the call to lazy-seq at the last line from the bottom. This is what replaces the recursive (in this case implicit recursion...notice I didn't use recur here) call with a lazy one. Without the recursion being wrapped in lazy-seq, there would be an eager evaluation, meaning that lazy-savings would be called again. And since there is no base-case as there is with the savings function above, it would eventually blow the call stack.
But why is this solution nicer? Because it allows you to do more interesting things with it. Whereas the function savings only returns a single value, lazy-savings returns a seq of values. Let's call it with the following:
user=> (take 10 (lazy-savings 0 4000 0.07))
You should get this:
(notice the interest rate is only .07 rather than 1.07)
See how you get all the values? If you wanted the final value, you could have called it with last.
(last (take 10 (lazy-savings 0 4000 0.07)))
Or if you wanted the 5th year, you could have either used (last (take 5 (lazy-savings 0 4000 0.07))) or you could have have done (nth (lazy-savings 0 4000 0.07) 5 ). In other words, having a lazily generated sequence is more powerful than the recursive version.
So the takeaway is two parts:
1) prefer laziness when possible
2) wrap the recursive part in lazy-seq, and don't worry about ending at the base case
Perhaps the last part bears a little more discussion. Recursion and induction are really two ways of looking at the same problem. Both involve a base case...a known condition. Generally, in an induction, you start with the base case and move forward, showing that if F(n-1) is true, then F(n) is also true (apologies to the math purists, as I can't recall if this is weak or strong induction). With recursion, you normally "walk backwards" to the base case. If for example you know that F(0) = 1, and you are solving for F(10), then you apply F(n-1) until you hit F(0) combining all the intermediate results.
So people who have been doing functional programming for awhile, and those more astute than I will probably go, "well, yeah...duh", but I just had a great realization that creating lazy sequences is really just the inverse of recursion. And what's the inverse of recursion? Induction.
So, I'm not going to go into what induction is exactly (whether it be mathematically speaking from a weak or strong perspective).
I decided to use a function I've written recursively several times that is of practical benefit (kinda) and turn it into a lazy sequence instead. Basically this is an interest calculator, given a certain starting principal, an amount that is saved (per year), and an interest. For my recursive solution, I also give a number of years, and in this case, I walk up to the number of years (starting with year 1). But for a lazy sequence, this isn't necessary. This is because, like induction, you don't try to hit the base case.
First, let's look at the recursive solution, then compare it to the lazy one.
(defn savings [ base saving interest years ] (loop [ b base y years ] (if (= y 0) b (do (println "year =" y "base = " b) (recur (* (+ b saving) interest) (dec y))))))
I did add a little print here so you can see how the interest rate accumulates. But this is your basic tail-call recursive solution to a problem like this. You would call it like this:
(savings 0 4000 1.07 10)
Which would calculate how much you would save in 10 years with a starting principal of 0 dollars, saving 4000 a year, and at a 1.07% total interest (yeah, pretty good I admit) for a total of 10 years.
But what about a lazy sequence solution? And why would you want a lazy solution anyway? Let's look at the lazy solution first. In some ways, it is easier.
(defn lazy-savings [ base savings interest ] (let [acc (* (+ savings base) interest) b (+ base acc savings) ] (lazy-seq (cons b (lazy-savings b savings interest)))))
Notice the call to lazy-seq at the last line from the bottom. This is what replaces the recursive (in this case implicit recursion...notice I didn't use recur here) call with a lazy one. Without the recursion being wrapped in lazy-seq, there would be an eager evaluation, meaning that lazy-savings would be called again. And since there is no base-case as there is with the savings function above, it would eventually blow the call stack.
But why is this solution nicer? Because it allows you to do more interesting things with it. Whereas the function savings only returns a single value, lazy-savings returns a seq of values. Let's call it with the following:
user=> (take 10 (lazy-savings 0 4000 0.07))
You should get this:
(notice the interest rate is only .07 rather than 1.07)
See how you get all the values? If you wanted the final value, you could have called it with last.
(last (take 10 (lazy-savings 0 4000 0.07)))
Or if you wanted the 5th year, you could have either used (last (take 5 (lazy-savings 0 4000 0.07))) or you could have have done (nth (lazy-savings 0 4000 0.07) 5 ). In other words, having a lazily generated sequence is more powerful than the recursive version.
So the takeaway is two parts:
1) prefer laziness when possible
2) wrap the recursive part in lazy-seq, and don't worry about ending at the base case
Perhaps the last part bears a little more discussion. Recursion and induction are really two ways of looking at the same problem. Both involve a base case...a known condition. Generally, in an induction, you start with the base case and move forward, showing that if F(n-1) is true, then F(n) is also true (apologies to the math purists, as I can't recall if this is weak or strong induction). With recursion, you normally "walk backwards" to the base case. If for example you know that F(0) = 1, and you are solving for F(10), then you apply F(n-1) until you hit F(0) combining all the intermediate results.
Saturday, February 4, 2012
Implementing clojure...in D?
Well, if you read my last few posts, you know I've been looking at a system's programming language called D. This is kind of a jack-of-all trades programming language, but what I find interesting about D is many of the features that you don't see in other systems programming languages in the C family (C/C++/ObjectiveC). For example:
*lambdas
closures
nested functions
**const and immutable (this is actually more secure than Java's final)
tail call optimization (Java might get this in Java 8)
***concurrency support (though no MVCC STM)
garbage collection
lazy evaluation of function args
true float, complex and imaginary numbers (ok, this is in other C family languages)
* C++11x does have support for lambdas, not sure about closures
**C++'s const is a huge confusing pain. D's seems a little simplified
***C++11x concurrency support appears (on my cursory examination) to consist of old-school locks and mutexes albeit in a portable language native fashion
There's also work on a LLVM compiler for D. This got me to thinking that it might be feasible to implement Clojure in D. Having LLVM support would enable a JIT compiler for D code, just as clojure emits bytecode on the fly for the JVM. Having true TCO, a more bare metal approach, imaginary number support, real floating support, and even safer immutability might even give it a leg up on Clojure itself...just as PyPy is even faster than canonical cpython. Implementing Clojure in C or C++ would be much harder I think, due to those languages lack of certain features.
Now first off, I will be the first to admit that I don't have the brain power to begin such a project, though if someone else took up the mantle, I would gladly help. I simply don't know enough about compilers, automata theory, grammars, AST, lexers, parsers, scanners, etc to go about creating my own language. It's always been a dream of mine...but I just don't have enough knowledge on the subject to go about doing it. When I finally pay off all my original school loans and finally try to get my Master's degree, I'll think about this as a project. But for now, it's just a nice fantasy.
You might be asking why I don't think clojure, as-is, is good enough. While high-level languages are great for building applications that essentially just get, manipulate and update data in one form or another, when you have to get to the metal and talk to the OS, they really are not all that hot. When you have need to get at drivers or system information, high level languages like Java or C# (or even languages like python or ruby) will leave you feeling frustrated. Since I work as an SDET for a company that builds SAS controllers, I routinely have to deal with low-level issues at the driver level (or even firmware level...of course at that level, you're pretty much stuck with C/C++ or assembly).
While tools like SWIG or jnaerator helps, it leaves a lot to be desired. I would LOVE to have a language with the expressive power and flexibility of Clojure, but with the ability to do low level calls with our many C/C++ libraries. Yes, I am aware of JNA, bridj, and SWIG. I've even played with HawtJNI a little. While they are nice, dealing with callbacks or going the opposite direction (from C calling Java) is problematic. That's why hand rolling JNI code, despite its difficulty, is in some ways still the best option.
Now admittedly, D doesn't natively understand header files, so it won't be a drop in replacement. But since D understands the same data types, it doesn't look like too much of a stretch to convert header files to D (though admittedly, tedious). For example, Java's lack of unsigned data types kills me when I do JNI (not to mention how much of a pain it is). Python's ctypes is probably the easiest of the high-level languages to muck around with C shared libraries, but it is of course slow (though PyPy is helping in that area enormously).
This idea really has my brain itching, and I wish I knew more about how to get started (not to mention have the time to do it). Not only would I have to learn all the aforementioned things about automata and compiler theory, I would have to basically become a guru in D and Clojure. I've only scratched the surface of Clojure (I still haven't played with protocols or multimethods, and I've only made one toy macro). And I am just now starting to learn D, and I can't wait for the book by Alexandrescu to arrive.
UPDATE: A thought just occurred to me. I could start writing some of the persistent data structures in D, like clojure's persistent data structures. This is something I could probably do now, and it would help solidify my understanding of data structures again. So I am going to go about creating D versions of persistent maps, lists, etc. I'll have to think about the sequence interface, and how I would implement that in D. A wrinkle is that Andrei wrote about the disadvantages of only doing forward iterative algorithms (like clojure's seqs). But I did see some examples in D of creating lazy containers, so at least I know it's possible to implement.
For reference, I will be looking at the clojure source code, and these:
Videocast of Rich Hickey on data structures
MIT's OpenCourseWare class that has a section on persistent structures
Andrei's article on using ranges instead of iterators
*lambdas
closures
nested functions
**const and immutable (this is actually more secure than Java's final)
tail call optimization (Java might get this in Java 8)
***concurrency support (though no MVCC STM)
garbage collection
lazy evaluation of function args
true float, complex and imaginary numbers (ok, this is in other C family languages)
* C++11x does have support for lambdas, not sure about closures
**C++'s const is a huge confusing pain. D's seems a little simplified
***C++11x concurrency support appears (on my cursory examination) to consist of old-school locks and mutexes albeit in a portable language native fashion
There's also work on a LLVM compiler for D. This got me to thinking that it might be feasible to implement Clojure in D. Having LLVM support would enable a JIT compiler for D code, just as clojure emits bytecode on the fly for the JVM. Having true TCO, a more bare metal approach, imaginary number support, real floating support, and even safer immutability might even give it a leg up on Clojure itself...just as PyPy is even faster than canonical cpython. Implementing Clojure in C or C++ would be much harder I think, due to those languages lack of certain features.
Now first off, I will be the first to admit that I don't have the brain power to begin such a project, though if someone else took up the mantle, I would gladly help. I simply don't know enough about compilers, automata theory, grammars, AST, lexers, parsers, scanners, etc to go about creating my own language. It's always been a dream of mine...but I just don't have enough knowledge on the subject to go about doing it. When I finally pay off all my original school loans and finally try to get my Master's degree, I'll think about this as a project. But for now, it's just a nice fantasy.
You might be asking why I don't think clojure, as-is, is good enough. While high-level languages are great for building applications that essentially just get, manipulate and update data in one form or another, when you have to get to the metal and talk to the OS, they really are not all that hot. When you have need to get at drivers or system information, high level languages like Java or C# (or even languages like python or ruby) will leave you feeling frustrated. Since I work as an SDET for a company that builds SAS controllers, I routinely have to deal with low-level issues at the driver level (or even firmware level...of course at that level, you're pretty much stuck with C/C++ or assembly).
While tools like SWIG or jnaerator helps, it leaves a lot to be desired. I would LOVE to have a language with the expressive power and flexibility of Clojure, but with the ability to do low level calls with our many C/C++ libraries. Yes, I am aware of JNA, bridj, and SWIG. I've even played with HawtJNI a little. While they are nice, dealing with callbacks or going the opposite direction (from C calling Java) is problematic. That's why hand rolling JNI code, despite its difficulty, is in some ways still the best option.
Now admittedly, D doesn't natively understand header files, so it won't be a drop in replacement. But since D understands the same data types, it doesn't look like too much of a stretch to convert header files to D (though admittedly, tedious). For example, Java's lack of unsigned data types kills me when I do JNI (not to mention how much of a pain it is). Python's ctypes is probably the easiest of the high-level languages to muck around with C shared libraries, but it is of course slow (though PyPy is helping in that area enormously).
This idea really has my brain itching, and I wish I knew more about how to get started (not to mention have the time to do it). Not only would I have to learn all the aforementioned things about automata and compiler theory, I would have to basically become a guru in D and Clojure. I've only scratched the surface of Clojure (I still haven't played with protocols or multimethods, and I've only made one toy macro). And I am just now starting to learn D, and I can't wait for the book by Alexandrescu to arrive.
UPDATE: A thought just occurred to me. I could start writing some of the persistent data structures in D, like clojure's persistent data structures. This is something I could probably do now, and it would help solidify my understanding of data structures again. So I am going to go about creating D versions of persistent maps, lists, etc. I'll have to think about the sequence interface, and how I would implement that in D. A wrinkle is that Andrei wrote about the disadvantages of only doing forward iterative algorithms (like clojure's seqs). But I did see some examples in D of creating lazy containers, so at least I know it's possible to implement.
For reference, I will be looking at the clojure source code, and these:
Videocast of Rich Hickey on data structures
MIT's OpenCourseWare class that has a section on persistent structures
Andrei's article on using ranges instead of iterators
Friday, February 3, 2012
Using emacs and leiningen on Windows for clojure pt. 3: Getting SLIME'd
(defproject MyFirstCljProject "1.0.0-SNAPSHOT" :description "FIXME: write description" :dependencies [[org.clojure/clojure "1.3.0"] [org.jboss.netty/netty "3.2.6.Final"] [ch.qos.logback/logback-core "0.9.30"] [ch.qos.logback/logback-classic "0.9.30"] [org.slf4j/slf4j-api "1.6.3"]])
Let's add our first source file. Notice in your MyFirstCljProject, there's a src directory, and under that, there is a MyFirstCljProject directory. Leiningen is kind of like Maven in that the src directory is the root folder for your package. Leiningen, by default through the new command, created your top level MyFirstCljProject directory. This is the first level element to your clojure package. Still confused?
Let's say you wanted to create a namespace like:
MyFirstCljProject.jinspector
That means you would find a ../src/MyFirstCljProject/jinspector.clj file. As another example, imagine you wanted a namespace of tools.networking.netty-client. When you have - in the namespace, you MUST have an underscore in the actual name of the file. So you would have a folder path of:
../src/tools/networking/netty_client.clj.
So now that we've got some namespacing under our belt, let's actually write that file. Let's create our own version of a Java class inspector. There is a clojure-contrib.repl-utils which does what I'll do here, but this is just for illustrative purposes. Open a file in emacs by using C-x C-f, and opening the C:/users/your_name/MyFirstCljProject/src/MyFirstCljProject/jinspector.clj
(ns MyFirstCljProject.jinspector)
(defn inspect [ x ] (let [cl (class? x) x (.getClass x) fields (.getFields cl) pfields (.getDeclaredFields cl) methods (.getMethods cl) pmethods (.getDeclaredMethods cl) p (.getSuperclass cl) ] (doseq [ x (concat fields members) ] (println x))))
Now that we have our method, let's actually try and use it...from SLIME! The first thing we have to do is actually start our project. Now, I've had some trouble using the swank-clojure command, 'clojure-jack-in'. This command is supposed to allow you to automatically connect to a leiningen project with a slime interface. Unfortunately, I haven't been able to get it to work.
However, you can start it manually. While your jinspector.clj buffer is the active one, do a M-x elein-run-task, and when it asks you for the task, enter 'swank'. Once you do that, you should have something like this:
Notice how the new buffer says "Connection opened on localhost port 4005"? That's our queue that we can use slime to connect to the swank-clojure plugin. You can do that with the M-x slime-connect command. It will ask you for an IP address (use the default 127.0.0.1...that's your local machine), and also hit enter again to use the default port of 4005. Once you do that, it should look like this:
Now we can begin playing with the clojure REPL!!
Looking at D language again.
Seriously, I need to stop reading so much on the web. I've got the attention span of a 5yr old in a candy store when it comes to programming languages. Somehow, I don't know how, I stumbled upon GLFW, which is yet another library aiming to help with OpenGL (like libSDL or freeglut or SFML). It turns out it natively supports D, and that got me to thinking. One thing led to another, and I came across an interesting article on concu rrency support in D by the renowned author and C++ guru Andrei Alexandrescu.
Another interesting aspect I discovered is the immutable support in D, and how it seems to mirror Clojure's default immutability (although in D, things aren't immutable by default...kind of like Scala, and that lack of default of default immutability has led people to call Scala not truly functional). However, while Clojure uses Software Transactional Memory as its way of avoiding ugly locks to memory access, D is using a different approach to its memory model. For example, by default, threads do not share data, and this has to be explicitly made so. It seems that D is using the more tried and true message-passing approach made most popular by erlang. It also appears that at least for the PyPy developers, that STM may able to remove the GIL in Python, so I wonder if the message passing model is the best to use, given that clojure is using STM, and even Scala is adding a STM approach to concurrency above and beyond their actor model.
So now I am all the more interested in D. It's basically a more modern alternative to C/C++...even C++11x. While C++11x does natively support threads, it still seems to be using the traditional, "lock memory access with mutexes" approach. Templates seem easier, and of course, there's built in garbage collection.
Since I am just starting, I am seriously considering using D and GLFW instead of C++ and SDL. SDL does look more advanced than GLFW however. Plus, as I mentioned before, I don't want my C++ skills to rust away. But D just looks a LOT nicer than C++ now.
Another interesting aspect I discovered is the immutable support in D, and how it seems to mirror Clojure's default immutability (although in D, things aren't immutable by default...kind of like Scala, and that lack of default of default immutability has led people to call Scala not truly functional). However, while Clojure uses Software Transactional Memory as its way of avoiding ugly locks to memory access, D is using a different approach to its memory model. For example, by default, threads do not share data, and this has to be explicitly made so. It seems that D is using the more tried and true message-passing approach made most popular by erlang. It also appears that at least for the PyPy developers, that STM may able to remove the GIL in Python, so I wonder if the message passing model is the best to use, given that clojure is using STM, and even Scala is adding a STM approach to concurrency above and beyond their actor model.
So now I am all the more interested in D. It's basically a more modern alternative to C/C++...even C++11x. While C++11x does natively support threads, it still seems to be using the traditional, "lock memory access with mutexes" approach. Templates seem easier, and of course, there's built in garbage collection.
Since I am just starting, I am seriously considering using D and GLFW instead of C++ and SDL. SDL does look more advanced than GLFW however. Plus, as I mentioned before, I don't want my C++ skills to rust away. But D just looks a LOT nicer than C++ now.
Tuesday, January 31, 2012
C++11x kinda neat but...
So I have been futzing around with porting some of the OpenGL SuperBible examples to use SDL instead of GLUT. It turns out the new SDL 1.3 is using the zlib license which is much more permissive, and it also can create OpenGL 3.x+ GL contexts.
So I started writing some C++ wrappers to help instantiate SDL and create a GL context for me. I forgot how atrocious C++ is. I haven't seriously written any "real" C++ code in about 4 years (I wrote some C++ which basically looked like C, but I didn't use Templates, namespaces, or classes). I even forgot how to pass in arguments to the parent class constructor. And I am aleady dreading the whole Copy Constructor vs. = operator overloading, using smart pointers, etc etc.
But the thing that really confuses me? Makefiles. I hate them, so I've been looking for an alternative. I found waf a long time ago, and I think I'll use it, but it looks like it will have a pretty steep learning curve too. But what I like about it is that it is more generic. It isn't JUST a C/C++ build/configuration tool. It's kind of interesting how the waf author handles dependencies. Although obviously he had to use a DAG to traverse dependencies, the python code he used to automatically determine these is pretty cool, and I wish I had thought of that for the code I did at work.
So I've already begun the tedious process of learning waf, in addition to re-learning C++, the C++11x additions, libSDL and of course OpenGL. Oh, did I mention that the game logic is in Clojure and will do low-level network communication with Netty (on the clojure side) to a boost.asio asynchronous network? To say that I have a lot to learn is a huge understatement.
Speaking of C++11x, some of the features seem interesting, but some of them I also just don't understand. I'm still trying to wrap my head around the ability to bind to rvalues. I am looking forward to auto and decltype, which may help me overcome my fear of Template programming. But still, I had to wonder...was there a better way?
So I started looking into Google's Go, and the D programming language. Go seems to be quite a departure from C/C++, eliminating some strongly typed aspects of the language. The syntax is also a little funkier, but nothing too bad I think. But I've already heard some people say that it's not exactly true that Go is a System Programming language, but rather a network programming language. It's also still in a very early stage of development. D on the other hand seems to be more of a real programming language. It also seems to do everything C++11x does, including a few that C++11x doesn't. In fact, the more I looked at D, the more I liked it. For example, it does automatic garbage collection (Go does too btw), it has an easier to read Template system, it has lambdas and closures, it has built-in complex numbers, built in unit-testing, and a few other nice features.
But...it has one major drawback (Go suffers the same fate). D does not have a preprocessor and thus does not understand .h header files. Sure, it can natively call C API libraries..but you have to define the types yourself. So basically, you have to convert header files into a D module. uggghh. I've been through this before with python ctypes, JNA, and bridj. It's not fun. And SWIG isn't much better. With SWIG, you still have to make a .i file, and even if you use a "raw" header, the stuff it outputs is barely human legible.
I'd love to use D, but I'd have to convert libSDL to modules (already done with the Derelict module...but it's using the 1.2 version of SDL which doesn't do OpenGL 3+ contexts, and I don't know what version OpenGL it supports). Sadly, if a system's programming language isn't compatible with C (and I don't just mean being able to call functions in a library, I mean to understand all the types in a header file too), then it's just not going to find a lot of uptake.
So I started writing some C++ wrappers to help instantiate SDL and create a GL context for me. I forgot how atrocious C++ is. I haven't seriously written any "real" C++ code in about 4 years (I wrote some C++ which basically looked like C, but I didn't use Templates, namespaces, or classes). I even forgot how to pass in arguments to the parent class constructor. And I am aleady dreading the whole Copy Constructor vs. = operator overloading, using smart pointers, etc etc.
But the thing that really confuses me? Makefiles. I hate them, so I've been looking for an alternative. I found waf a long time ago, and I think I'll use it, but it looks like it will have a pretty steep learning curve too. But what I like about it is that it is more generic. It isn't JUST a C/C++ build/configuration tool. It's kind of interesting how the waf author handles dependencies. Although obviously he had to use a DAG to traverse dependencies, the python code he used to automatically determine these is pretty cool, and I wish I had thought of that for the code I did at work.
So I've already begun the tedious process of learning waf, in addition to re-learning C++, the C++11x additions, libSDL and of course OpenGL. Oh, did I mention that the game logic is in Clojure and will do low-level network communication with Netty (on the clojure side) to a boost.asio asynchronous network? To say that I have a lot to learn is a huge understatement.
Speaking of C++11x, some of the features seem interesting, but some of them I also just don't understand. I'm still trying to wrap my head around the ability to bind to rvalues. I am looking forward to auto and decltype, which may help me overcome my fear of Template programming. But still, I had to wonder...was there a better way?
So I started looking into Google's Go, and the D programming language. Go seems to be quite a departure from C/C++, eliminating some strongly typed aspects of the language. The syntax is also a little funkier, but nothing too bad I think. But I've already heard some people say that it's not exactly true that Go is a System Programming language, but rather a network programming language. It's also still in a very early stage of development. D on the other hand seems to be more of a real programming language. It also seems to do everything C++11x does, including a few that C++11x doesn't. In fact, the more I looked at D, the more I liked it. For example, it does automatic garbage collection (Go does too btw), it has an easier to read Template system, it has lambdas and closures, it has built-in complex numbers, built in unit-testing, and a few other nice features.
But...it has one major drawback (Go suffers the same fate). D does not have a preprocessor and thus does not understand .h header files. Sure, it can natively call C API libraries..but you have to define the types yourself. So basically, you have to convert header files into a D module. uggghh. I've been through this before with python ctypes, JNA, and bridj. It's not fun. And SWIG isn't much better. With SWIG, you still have to make a .i file, and even if you use a "raw" header, the stuff it outputs is barely human legible.
I'd love to use D, but I'd have to convert libSDL to modules (already done with the Derelict module...but it's using the 1.2 version of SDL which doesn't do OpenGL 3+ contexts, and I don't know what version OpenGL it supports). Sadly, if a system's programming language isn't compatible with C (and I don't just mean being able to call functions in a library, I mean to understand all the types in a header file too), then it's just not going to find a lot of uptake.
Tuesday, January 24, 2012
Quick update
Wow, long time no post. Been busy over the holidays so I haven't had a chance to do much coding. Doesn't help that I bought a ton of old Twilight 2000 pdf's from rpgnow.com to keep me busy reading and reminiscing. Hopefully this week, I'll put up the third post in the Emacs + Clojure + Leiningen series.
I have however lately been thinking about where to spend my free time in programming. I still want to do some clojure programming, and I will, but the pragmatic side of me also wants to get back into some C/C++ programming. I've slowly been reading the OpenGL SuperBible book (almost done with Chapter 4), and my original idea was that I would try to port the C++ code to clojure code (via the lwjgl library). But I realized a couple things...
1) The graphics side would be slow
2) I don't want my C/C++ skills to rust away
3) Might be an opportunity to pick up some C++11x
4) I won't have to do any mental conversion of the code
The disadvantage of course is that I don't want all of my game to use C++. I want the graphics side to be in C++ and possibly some OpenCL, but the actual game logic to be in Clojure. So how am I going to accomplish that? I could write a whole bunch of JNI so that the C++ code could call Java code. Another option would be to do some IPC, and have the C++ code shuttle information and make method calls via some kind of RPC mechanism.
I already have a little bit of experience with Netty, and I even started writing my own messaging protocol to be communicated over the socket. Although Netty was relatively painless, writing TCP socket programming in C/C++ is not fun...not to mention asynchronous or multi-threaded socket programming. I could use boost's ASIO library, but that in itself looks like a pretty big learning curve.
So I'm still debating what to do. JNI is tedious and error prone (usually requiring passing NIO byte buffers all over the place), and socket programming isn't fun. I think I'll go the IPC socket route...but I am kind of dreading it.
But my decision to use C++ will be a time eater. Not only will I have to refresh my skills (and pick up some of the new C++11x features), there's a ton of new libraries I will have to figure out like libSDL, boost.asio, boost.threads, and of course OpenGL itself. I also want to learn a new build system called waf. Makefiles are horrible, and I don't want to have specific IDE project files for each OS. In a nutshell, waf is a replacement for autotools like functionality, but it is a bit complex. That being said, I think it's approach seems more reasonable than other tools like cmake or rake.
But, a thousand mile journey begins with the first step. I've already cloned the latest libSDL 1.3, built it, and make a simple OpenGL context window with it. This will allow me to use SDL instead of GLUT that the OpenGL SuperBible book uses. A small step, but a step none the less.
I have however lately been thinking about where to spend my free time in programming. I still want to do some clojure programming, and I will, but the pragmatic side of me also wants to get back into some C/C++ programming. I've slowly been reading the OpenGL SuperBible book (almost done with Chapter 4), and my original idea was that I would try to port the C++ code to clojure code (via the lwjgl library). But I realized a couple things...
1) The graphics side would be slow
2) I don't want my C/C++ skills to rust away
3) Might be an opportunity to pick up some C++11x
4) I won't have to do any mental conversion of the code
The disadvantage of course is that I don't want all of my game to use C++. I want the graphics side to be in C++ and possibly some OpenCL, but the actual game logic to be in Clojure. So how am I going to accomplish that? I could write a whole bunch of JNI so that the C++ code could call Java code. Another option would be to do some IPC, and have the C++ code shuttle information and make method calls via some kind of RPC mechanism.
I already have a little bit of experience with Netty, and I even started writing my own messaging protocol to be communicated over the socket. Although Netty was relatively painless, writing TCP socket programming in C/C++ is not fun...not to mention asynchronous or multi-threaded socket programming. I could use boost's ASIO library, but that in itself looks like a pretty big learning curve.
So I'm still debating what to do. JNI is tedious and error prone (usually requiring passing NIO byte buffers all over the place), and socket programming isn't fun. I think I'll go the IPC socket route...but I am kind of dreading it.
But my decision to use C++ will be a time eater. Not only will I have to refresh my skills (and pick up some of the new C++11x features), there's a ton of new libraries I will have to figure out like libSDL, boost.asio, boost.threads, and of course OpenGL itself. I also want to learn a new build system called waf. Makefiles are horrible, and I don't want to have specific IDE project files for each OS. In a nutshell, waf is a replacement for autotools like functionality, but it is a bit complex. That being said, I think it's approach seems more reasonable than other tools like cmake or rake.
But, a thousand mile journey begins with the first step. I've already cloned the latest libSDL 1.3, built it, and make a simple OpenGL context window with it. This will allow me to use SDL instead of GLUT that the OpenGL SuperBible book uses. A small step, but a step none the less.
Subscribe to:
Posts (Atom)