Well, the name of the blog IS Under a Boddhi Tree, so here goes my first non-software engineering post.
The recent political theatre and downgrading of America's credit rating has made me become upset at what I see as ignorance, greed, and political showmanship over the welfare of the nation and its people. I couldn't fathom the levels of ignorance that I also saw regarding facts on the situation. For example, a Yahoo Finance News web report consistently said that S&P downgraded America's credit due to not enough spending cuts. However, if you actually read the S&P report, you will see that it said that there should have been a balance of spending cuts and raising revenues.
Unfortunately, people see only what they want to see, and people hear only what they want to hear. If they are conservatives, they only want to hear statements supporting their beliefs (no new taxes, ending social welfare programs, government doesn't create jobs, illegal immigrants are taking our jobs, gay marriage is destroying our culture blah blah), and if you are a progressive or liberal, you only want to hear what you believe in (universal medical care, less need for a strong military, businesses need strong regulations, fair taxation, equal rights for all races, sex orientation, unions only protect workers, they never create unrealistic salaries, freedom of or from beliefs, etc etc). This habit for people to see only what they want is a huge reason for all suffering. In fact, I might say that the whole purpose of Buddhism is just to see things as they are, with no filters, and no expectations.
I have always considered myself pretty open-minded. When I was younger, I was a Republican, now I am a Democrat. Also when I was younger, I was an empirical material objectivist (if you couldn't sense it with your 5 senses, it didn't exist), but now I am an idealist. But I am beginning to see something now....these very labels are wrong. I really should have known this a long time ago due to the fact that I am of mixed ethnicity (Irish, German, and Polish from my dad, and Filipino, Malaysian, Chinese, Spanish, Persian from my mom). I do not consider myself white or asian...I am just a human being.
And that's the point. I am not a progressive. I am just someone who is trying to see the world as it truly is, not as I want or hope it should be. It is very hard. I feel like the world SHOULD be a certain way. But the fact is, I don't even truly know who I am. And if I don't know who I am, how am I supposed to be a judge for the way the world should be? Does that sound strange that I don't know who or what I am? People who have not practiced eastern religions are often confounded by that statement. They don't even really understand the question I think.
But asking, discovering, and knowing who and what you truly are is, I think, the true path to awakening. I am definitely not a label. I am not a progressive, or a democrat, or a computer scientist or engineer. These are facets perhaps, but they must be dropped. Letting go of these views of my 'self' helps me understand who I truly am. Otherwise, these views separate me from knowing reality as it is. I have tried to tell others before that is not your belief in Christianity, Judaism, Islam, Hinduism, Conservatism, Liberalism, Socialism, Capitalism, etc that is wrong. It is belief in and of itself that is wrong. If you can not know it and understand it for yourself, then it is an illusion, a desire that ignores reality for fantasy instead.
So who am I? I do not yet know, but I have traveled a little bit across the stream. I have understood that I am not my thoughts or feelings. I am still groping in the dark, but at least I know that I am in the dark. And I am beginning to see that getting mad at all this political theatre is pointless. Trying to get people to understand me is pointless. Trying to convince others about anything is also pointless. I know all about Maslowe's hierarchy, but chasing after all these "should be's" and "I want's" is in itself suffering.
All that matters is being aware, seeing how things are, and discovering what we truly are.
Saturday, August 6, 2011
Saturday, July 23, 2011
Got my remote process server working.
So I made a few changes, and now my client can pick up the stdout of a remote process, disconnect from it, and reattach if need be. The first problem I ran into was properly passing in the Channel object that got generated when a connection between the client and server was made. This also had the added effect that the process on the remote machine does not actually start until the connection was made.
The thing that really threw me though was why once I made the above change, the client wasn't receiving anything over the channel. At first I thought maybe I had to make a netty ChannelFuture object, due to the asynchronous nature of netty itself.
It is however working now, including the ability to send a few commands (that go over the channel from the client to the server) to the remote process's standard input. I can now edit the command string, show the command, kill the process, or restart it. I still need to work on changing the environment variables or working directory however. The other big limitation, and one that I don't think I can or will workaround is the ability to log in as a different user.
This is where SSH shines. With SSH you can log in as a specific user and get all the permissions that user has. My server will run with the same user ID and permissions as whomever launched the initial TaskServer process. This has its cons of course, but still, I think this is ok.
The next thing I'll do is add the ability to run multiple processes per TaskServer. The trick here will be encoding the output from the different processes in such a way that the client can piece together what output came from what process. The most obvious thought that comes to mind is to simply append some key to the line of output (since I am using netty's line delimiter encoder/decoder). This would require the ability to filter the incoming line to the client, and I think that could be a speed issue. A second option could be to not use line delimiter, and roll my own encoder/decoder, but that will be quite a bit more work.
Still, I think the little server is kinda cool. I still need to officially give it a license, and I've been leaning towards the Eclipse Public License. I'm also considering switching to using Git, and having it hosted on github or some other service. I'm starting to learn git for a couple of reasons:
1. It appears to be more popular than mercurial
2. Using jgit, it is easy to provide as a client
3. I want to use gerrit for code reviews
Many projects appear to be moving towards git. Eclipse now uses egit/jgit as the default Team provider now (instead of SVN). And although Sun seemed fond of mercurial, many java projects, are using git. Clojure and Android are two popular ones that are of interest to me using git now. Git does seem more complicated than mercurial, not that I'm an advanced user of mercurial by any means. But having this project using git will help me learn git as well.
The thing that really threw me though was why once I made the above change, the client wasn't receiving anything over the channel. At first I thought maybe I had to make a netty ChannelFuture object, due to the asynchronous nature of netty itself.
It is however working now, including the ability to send a few commands (that go over the channel from the client to the server) to the remote process's standard input. I can now edit the command string, show the command, kill the process, or restart it. I still need to work on changing the environment variables or working directory however. The other big limitation, and one that I don't think I can or will workaround is the ability to log in as a different user.
This is where SSH shines. With SSH you can log in as a specific user and get all the permissions that user has. My server will run with the same user ID and permissions as whomever launched the initial TaskServer process. This has its cons of course, but still, I think this is ok.
The next thing I'll do is add the ability to run multiple processes per TaskServer. The trick here will be encoding the output from the different processes in such a way that the client can piece together what output came from what process. The most obvious thought that comes to mind is to simply append some key to the line of output (since I am using netty's line delimiter encoder/decoder). This would require the ability to filter the incoming line to the client, and I think that could be a speed issue. A second option could be to not use line delimiter, and roll my own encoder/decoder, but that will be quite a bit more work.
Still, I think the little server is kinda cool. I still need to officially give it a license, and I've been leaning towards the Eclipse Public License. I'm also considering switching to using Git, and having it hosted on github or some other service. I'm starting to learn git for a couple of reasons:
1. It appears to be more popular than mercurial
2. Using jgit, it is easy to provide as a client
3. I want to use gerrit for code reviews
Many projects appear to be moving towards git. Eclipse now uses egit/jgit as the default Team provider now (instead of SVN). And although Sun seemed fond of mercurial, many java projects, are using git. Clojure and Android are two popular ones that are of interest to me using git now. Git does seem more complicated than mercurial, not that I'm an advanced user of mercurial by any means. But having this project using git will help me learn git as well.
Wednesday, June 29, 2011
Piping stdout and stdin to/from a socket
I'm kind of surprised no one has tried doing this before...or at least I haven't found anyone attempting to do this. Perhaps it's too closely related to SSH for people to bother. Basically, what I am trying to do is create a class (in Java) that launches a subprocess, and in which the stdout, stderr and stdin of the process are all linked to a network socket. The key advantage to this over SSH is that unlike SSH, if you close your terminal, you don't also kill your process(es) (though I'm not sure if there's already a way to do this in SSH). And of course, this will be applicable to windows platform.
I'm using the Netty project currently as my low level NIO network framework. It's not too bad, but I do seem to be having some trouble getting the asynchronous event driven nature of Netty working with the thread reading mechanic of the stdout and stdin reader of the subprocess. Basically, in order to read the stdout of the subprocess, I have a separate thread running which is constantly checking to see if there's anything in the process's InputStream (yeah, confusing...go look at the java.lang.Process class, but stdout is actually an InputStream).
So what I have to do is somehow, in the Runnable's run() method, pass Netty's Channel object to the Thread object and write the output to the channel. I can currently see when a client connects to the server, and the server can send a basic message, but I can't seem to pass the (shared) channel object (which I assign when the channel is connected) and have the stdout reader thread be able to use it.
Still, I feel like I'm on the right track. And since I think this could be a useful project, I'll open source this. Eventually, if I ever get it working, I'll port it to clojure. I'd eventually like to add a security layer (Netty has SSL support). But first things first.
I'm using the Netty project currently as my low level NIO network framework. It's not too bad, but I do seem to be having some trouble getting the asynchronous event driven nature of Netty working with the thread reading mechanic of the stdout and stdin reader of the subprocess. Basically, in order to read the stdout of the subprocess, I have a separate thread running which is constantly checking to see if there's anything in the process's InputStream (yeah, confusing...go look at the java.lang.Process class, but stdout is actually an InputStream).
So what I have to do is somehow, in the Runnable's run() method, pass Netty's Channel object to the Thread object and write the output to the channel. I can currently see when a client connects to the server, and the server can send a basic message, but I can't seem to pass the (shared) channel object (which I assign when the channel is connected) and have the stdout reader thread be able to use it.
Still, I feel like I'm on the right track. And since I think this could be a useful project, I'll open source this. Eventually, if I ever get it working, I'll port it to clojure. I'd eventually like to add a security layer (Netty has SSL support). But first things first.
Thursday, June 16, 2011
Functional python
Sadly, I don't have an opportunity to write clojure at work, but I am able to write in python, so I've been tackling some common problems in python in a more functional style. I've discovered that list comprehension, lambdas, map, and reduce are your friends. Also, writing in a functional style often means writing less iterative code, and more recursive code.
So first things first, how can list comprehensions help? Imagine if you a list of characters, and you want to combine all of them into a word. Of course the pythonic way to do this would be:
example = [ 'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd']
"".join(example)
This is of course perfectly valid. A functional approach to this would be this:
reduce(lambda x,y: x + y, example)
In this case, the lambda is adding (concatenating) two strings together. The reduce function takes the first two arguments, and then applies the result of that to the next argument.
Using the map() function is also handy, and can sometimes be easier to read then list comprehensions. For example, the two below are equivalent:
[ x**2 for x in (1,2,3,4) ]
map(lambda x: x**2, (1,2,3,4))
However, what if you wanted to multiply some_collection[x] + another_collection[x]? If you try to do this as a list comprehension, you won't get what you think:
[ x * y for x in (1,2,3,4) for y in (10,20,30,40) ] ## try it
Instead, you can use a map here:
map(lambda x,y: x*y, [1,2,3,4], [10,20,30,40])
While the examples above have been relatively trivial, here's a more complicated scenario. Imagine that you are given an amount of change, and you are to calculate all the possible combinations of quarters, dimes, nickels and pennies you can get. For example, if you are given 27 cents, you could have:
1 quarter, 2 pennies
2 dimes, 1 nickel, 2 pennies
1 dime, 2 nickels, 2 pennies
etc etc.
So how would you go about doing this? When I first thought about this problem, I tackled it in the usual manner by trying to come up with an iterative imperative solution. But I later decided (after the problem being fresh out of my mind) to come up with a recursive solution. However, it's not only recursive, it's a mutual recursive problem.
So think about the problem like this.
1. I have a total. Given a number of quarters Q, I know the remainder of change (total - (25 * Q) )
2. I have a remainder. Given a number of dimes D, I know the remainder of change ( remainder - (10 * D))
3. I have a remainder. Given a number of nickels N, I know the remainder of change ( remainder - (5 * N))
4. Any remainder left must be pennies
Do you see how all the problems are similar? there are some gotchas however. But below is the code representing my solution to this tricky problem. I used a list here as return values so I could add lists together when one function call popped off the stack.
import pprint
def remainderNickels(total, nickels):
if (nickels * 5) < total:
newn = nickels + 1
return remainderNickels(total, newn)
else:
return [ {'pennies' : total - (5 * (nickels - 1)), 'nickels' : nickels - 1 }]
def remainderDimes(total, dimes):
remainder = total - (dimes*10)
if (dimes * 10) < total:
newd = dimes + 1
if remainder >= 5:
return remainderDimes(total, newd) + [ { 'remainder' : remainderNickels(remainder, 1), 'dimes' : dimes }]
else:
return [ {'pennies' : total - (10 * dimes), 'dimes' : dimes }]
else:
return [[]]
def remainderQuarters(total, quarters):
remainder = total - (quarters*25)
if (quarters * 25) < total:
newq = quarters + 1
if remainder >= 10:
return remainderQuarters(total, newq) + [{ 'remainder' : remainderDimes(remainder, 1), 'quarters' : quarters} ]
elif remainder >= 5:
return remainderQuarters(total, newq) + [{ 'remainder' : remainderNickels(remainder, 1), 'quarters' : quarters} ]
else:
print "remainder = ", remainder
return [ {'pennies' : remainder, 'quarters' : quarters }]
else:
return []
q = remainderQuarters(88, 1)
pp =pprint.PrettyPrinter()
pp.pprint(q)
So first things first, how can list comprehensions help? Imagine if you a list of characters, and you want to combine all of them into a word. Of course the pythonic way to do this would be:
example = [ 'h', 'e', 'l', 'l', 'o', ' ', 'w', 'o', 'r', 'l', 'd']
"".join(example)
This is of course perfectly valid. A functional approach to this would be this:
reduce(lambda x,y: x + y, example)
In this case, the lambda is adding (concatenating) two strings together. The reduce function takes the first two arguments, and then applies the result of that to the next argument.
Using the map() function is also handy, and can sometimes be easier to read then list comprehensions. For example, the two below are equivalent:
[ x**2 for x in (1,2,3,4) ]
map(lambda x: x**2, (1,2,3,4))
However, what if you wanted to multiply some_collection[x] + another_collection[x]? If you try to do this as a list comprehension, you won't get what you think:
[ x * y for x in (1,2,3,4) for y in (10,20,30,40) ] ## try it
Instead, you can use a map here:
map(lambda x,y: x*y, [1,2,3,4], [10,20,30,40])
While the examples above have been relatively trivial, here's a more complicated scenario. Imagine that you are given an amount of change, and you are to calculate all the possible combinations of quarters, dimes, nickels and pennies you can get. For example, if you are given 27 cents, you could have:
1 quarter, 2 pennies
2 dimes, 1 nickel, 2 pennies
1 dime, 2 nickels, 2 pennies
etc etc.
So how would you go about doing this? When I first thought about this problem, I tackled it in the usual manner by trying to come up with an iterative imperative solution. But I later decided (after the problem being fresh out of my mind) to come up with a recursive solution. However, it's not only recursive, it's a mutual recursive problem.
So think about the problem like this.
1. I have a total. Given a number of quarters Q, I know the remainder of change (total - (25 * Q) )
2. I have a remainder. Given a number of dimes D, I know the remainder of change ( remainder - (10 * D))
3. I have a remainder. Given a number of nickels N, I know the remainder of change ( remainder - (5 * N))
4. Any remainder left must be pennies
Do you see how all the problems are similar? there are some gotchas however. But below is the code representing my solution to this tricky problem. I used a list here as return values so I could add lists together when one function call popped off the stack.
import pprint
def remainderNickels(total, nickels):
if (nickels * 5) < total:
newn = nickels + 1
return remainderNickels(total, newn)
else:
return [ {'pennies' : total - (5 * (nickels - 1)), 'nickels' : nickels - 1 }]
def remainderDimes(total, dimes):
remainder = total - (dimes*10)
if (dimes * 10) < total:
newd = dimes + 1
if remainder >= 5:
return remainderDimes(total, newd) + [ { 'remainder' : remainderNickels(remainder, 1), 'dimes' : dimes }]
else:
return [ {'pennies' : total - (10 * dimes), 'dimes' : dimes }]
else:
return [[]]
def remainderQuarters(total, quarters):
remainder = total - (quarters*25)
if (quarters * 25) < total:
newq = quarters + 1
if remainder >= 10:
return remainderQuarters(total, newq) + [{ 'remainder' : remainderDimes(remainder, 1), 'quarters' : quarters} ]
elif remainder >= 5:
return remainderQuarters(total, newq) + [{ 'remainder' : remainderNickels(remainder, 1), 'quarters' : quarters} ]
else:
print "remainder = ", remainder
return [ {'pennies' : remainder, 'quarters' : quarters }]
else:
return []
q = remainderQuarters(88, 1)
pp =pprint.PrettyPrinter()
pp.pprint(q)
Monday, June 13, 2011
Things I need to get better at
Being a jack-of-all-trades suits me. I enjoy being able to dabble in many diverse areas of technology. Although my knowledge is shallow in many areas it is broad. I am a decent programmer in many languages, and I have in my short career worked with embedded device drivers and firmware, writing proprietary messaging protocols over TCP/IP sockets, stored and queried needed information on both MySQL and H2 databases, wrote XMLRPC servers, RMI servers, written JNI wrappers around C shared libraries, and many things inbetween. In other words, I have a pretty good view of the entire technology stack.
But there are still many areas I need improving on. Now that I am writing more java again, and I wish to be better at clojure, I need to get better at the java ecosystem. For example:
1. Get better at maven. I have built a few non-trivial maven projects, including one multi-module, but a lot of maven's finer points still elude me.
2. Javadoc commenting. I just let Eclipse auto-fill in the params and return values, but I really should know all the markup for it.
3. Annotations. I understand them in theory, but I've never written one (same goes for python decorators)
4. Unit testing. Yeah yeah, being in Test, I should know JUnit or TestNG like the back of my hand. While I realize their importance, sadly, time constraints often win. I have tried writing some TestNG unit tests, but they are not being called from maven, and I haven't had time to figure out why
5. OSGi. While I have written two eclipse plugins, I still don't truly understand a lot of OSGi. I understand what it's for (modularity to decrease coupling, and provide metadata to end jar hell), but it's such a huge beast that I need to know more
There's also a lot in general that I want to get better at or relearn:
1. C/C++. I haven't seriously written any C or C++ in a little over a year when I was doing some JNI wrapping. The new features in C++0x looks interesting, and eventually, I hope to get back to more JNI programming.
2. Advanced python. By this, I mean stuff like decorators, generators, continuations, and metaprogramming. I actually think I finally understand generators (functions that yield an iterator like object), and how they can be used for continuation style programming. I once showed a coworker how to implement "private" methods and fields in python through implementing some of the magic methods, but that's the most metaprogramming I've done.
3. Algorithm design. After implementing a homegrown software dependency installer program, I came up on my own a depth first search algorithm that could do post-order traversal. I didn't know it was called that until after I read the chapter on traversal algorithms in the book Python Algorithms: Mastering Basic Algorithms in the Python Language.
4. Concurrent programming: Most of my experience with multi-threading has simply been to spawn a new thread to prevent blocking during a long running task. Only twice have I had to share data across threads, and honestly, I'm not sure if I synchronized things right. One reason I wanted to learn clojure was for its approach to concurrent programming (even OpenGL is massively parallel in nature).
And while all of the above will help me in the "real" world, for my own personal desire, I still want to learn OpenGL, and get better at clojure. I also want to get better at JBoss's Netty NIO framework. Sadly, much of my spare time is spent working for work.
But there are still many areas I need improving on. Now that I am writing more java again, and I wish to be better at clojure, I need to get better at the java ecosystem. For example:
1. Get better at maven. I have built a few non-trivial maven projects, including one multi-module, but a lot of maven's finer points still elude me.
2. Javadoc commenting. I just let Eclipse auto-fill in the params and return values, but I really should know all the markup for it.
3. Annotations. I understand them in theory, but I've never written one (same goes for python decorators)
4. Unit testing. Yeah yeah, being in Test, I should know JUnit or TestNG like the back of my hand. While I realize their importance, sadly, time constraints often win. I have tried writing some TestNG unit tests, but they are not being called from maven, and I haven't had time to figure out why
5. OSGi. While I have written two eclipse plugins, I still don't truly understand a lot of OSGi. I understand what it's for (modularity to decrease coupling, and provide metadata to end jar hell), but it's such a huge beast that I need to know more
There's also a lot in general that I want to get better at or relearn:
1. C/C++. I haven't seriously written any C or C++ in a little over a year when I was doing some JNI wrapping. The new features in C++0x looks interesting, and eventually, I hope to get back to more JNI programming.
2. Advanced python. By this, I mean stuff like decorators, generators, continuations, and metaprogramming. I actually think I finally understand generators (functions that yield an iterator like object), and how they can be used for continuation style programming. I once showed a coworker how to implement "private" methods and fields in python through implementing some of the magic methods, but that's the most metaprogramming I've done.
3. Algorithm design. After implementing a homegrown software dependency installer program, I came up on my own a depth first search algorithm that could do post-order traversal. I didn't know it was called that until after I read the chapter on traversal algorithms in the book Python Algorithms: Mastering Basic Algorithms in the Python Language.
4. Concurrent programming: Most of my experience with multi-threading has simply been to spawn a new thread to prevent blocking during a long running task. Only twice have I had to share data across threads, and honestly, I'm not sure if I synchronized things right. One reason I wanted to learn clojure was for its approach to concurrent programming (even OpenGL is massively parallel in nature).
And while all of the above will help me in the "real" world, for my own personal desire, I still want to learn OpenGL, and get better at clojure. I also want to get better at JBoss's Netty NIO framework. Sadly, much of my spare time is spent working for work.
Sunday, May 22, 2011
Why Test Engineers need to be good developers too
It would seem that in many organizations, "automation" has become something of a buzz word. It's a term that makes upper level managers happy, but which often confound many, including Architects and lower level managers.
In order to achieve automation, especially on products which are GUI based (either as a thick client, or some web based mechanism), there is a tendency to want to use "capture/replay" tools. In essence, these are tools that are designed to capture mouse and keyboard clicks, save them, and replay it later. While this sounds great, and I am not against such a tool as part of the overall test strategy, when it is the SOLE method of testing, I am strongly against it.
I have read many sites talk about the dangers of using capture/replay tools, but they usually just harp on the difficulty of maintaining the "scripts" (the file that holds all the captured events). If the GUI interface changes, then the script becomes useless. While this is a valid point, it is not in my opinion the most serious shortcoming. The most serious shortcoming is that it does not rely on any knowledge about how the product works.
Remember that I said Test Engineer, and not Quality Assurance Engineer in the title of this blog. A QAE could rightfully say that he only does black box testing and should see the system as a customer would see the system. Test Engineers on the other hand should do some white box testing, and should be familiar of the inner workings of the system being tested. Why is this beneficial?
If you don't know how it's supposed to work, how are you supposed to know when you have a correct answer or not? I have often given an analogy that Test Engineers are teachers, and Developers are students. Test Engineers must grade what the Developers do. If the Test Engineer himself doesn't know what the answer is, that is akin to a teacher asking the student what the answer should be before the student takes the test. Letting developers have this level of control over the validity of a product is no different than letting the fox guard the hen house ("oh, that's 'working as designed', we MEANT for it to behave that way").
The second big reason you want Test Engineers to be developers and know how the system works is for first level triage. If your Test Engineers have no clue how the underlying system works, how can you expect them to debug it? Is the problem in the driver? The firmware? Maybe it's somewhere higher up in the stack, and it is a framework or application they are using. Or what if it's a low-level physical problem (jitter, noise, etc)? If all your Test Engineers do is execute tests from a Test Plan, without understanding to at least some level, the interactions of the various components, he will be ineffective at providing first level debugging, and this burden will fall to a developer. The danger here is that an ineffective defect assignment will waste a developer's time (for example, if the Test Engineer assumed the defect was firmware problem, but it turned out to be a driver problem).
And finally, how is a Test Engineer supposed to come up with new Test Cases or ad-hoc tests if he doesn't understand the system from a low-level perspective. At best, he can read a spec and determine what should be tested, but he won't necessarily know how to stress a feature, nor will he necessarily understand how to come up with invalid inputs (negative testing). Being able to "see" the weak points in a product requires knowing how that system works. Many famous martial arts masters of the past were also great healers of their time, because by understanding how the body worked, they were much better at knowing the weaknesses of the body. The same logic applies to testing.
To give you a real world example, suppose you want to use sg_utils to send generic SCSI commands. This is all well and good, but suppose something doesn't go right. Where was the problem? If you don't know what a CDB (command descriptor block) is, or how to read a SAS trace, what good will it do you? At best, you have thrown the problem over the wall to a developer to fix. This is ok for a QAE, but not a Test Engineer.
And as I mentioned in my previous blog, part of the reason Test Engineers get little respect is because they are seen by developers as little more than testers. Although Test Driven Development has gained some traction, notice that it focuses on developers writing better unit tests. The concept of Development Driven Testing has unfortunately not caught on.
In order to achieve automation, especially on products which are GUI based (either as a thick client, or some web based mechanism), there is a tendency to want to use "capture/replay" tools. In essence, these are tools that are designed to capture mouse and keyboard clicks, save them, and replay it later. While this sounds great, and I am not against such a tool as part of the overall test strategy, when it is the SOLE method of testing, I am strongly against it.
I have read many sites talk about the dangers of using capture/replay tools, but they usually just harp on the difficulty of maintaining the "scripts" (the file that holds all the captured events). If the GUI interface changes, then the script becomes useless. While this is a valid point, it is not in my opinion the most serious shortcoming. The most serious shortcoming is that it does not rely on any knowledge about how the product works.
Remember that I said Test Engineer, and not Quality Assurance Engineer in the title of this blog. A QAE could rightfully say that he only does black box testing and should see the system as a customer would see the system. Test Engineers on the other hand should do some white box testing, and should be familiar of the inner workings of the system being tested. Why is this beneficial?
If you don't know how it's supposed to work, how are you supposed to know when you have a correct answer or not? I have often given an analogy that Test Engineers are teachers, and Developers are students. Test Engineers must grade what the Developers do. If the Test Engineer himself doesn't know what the answer is, that is akin to a teacher asking the student what the answer should be before the student takes the test. Letting developers have this level of control over the validity of a product is no different than letting the fox guard the hen house ("oh, that's 'working as designed', we MEANT for it to behave that way").
The second big reason you want Test Engineers to be developers and know how the system works is for first level triage. If your Test Engineers have no clue how the underlying system works, how can you expect them to debug it? Is the problem in the driver? The firmware? Maybe it's somewhere higher up in the stack, and it is a framework or application they are using. Or what if it's a low-level physical problem (jitter, noise, etc)? If all your Test Engineers do is execute tests from a Test Plan, without understanding to at least some level, the interactions of the various components, he will be ineffective at providing first level debugging, and this burden will fall to a developer. The danger here is that an ineffective defect assignment will waste a developer's time (for example, if the Test Engineer assumed the defect was firmware problem, but it turned out to be a driver problem).
And finally, how is a Test Engineer supposed to come up with new Test Cases or ad-hoc tests if he doesn't understand the system from a low-level perspective. At best, he can read a spec and determine what should be tested, but he won't necessarily know how to stress a feature, nor will he necessarily understand how to come up with invalid inputs (negative testing). Being able to "see" the weak points in a product requires knowing how that system works. Many famous martial arts masters of the past were also great healers of their time, because by understanding how the body worked, they were much better at knowing the weaknesses of the body. The same logic applies to testing.
To give you a real world example, suppose you want to use sg_utils to send generic SCSI commands. This is all well and good, but suppose something doesn't go right. Where was the problem? If you don't know what a CDB (command descriptor block) is, or how to read a SAS trace, what good will it do you? At best, you have thrown the problem over the wall to a developer to fix. This is ok for a QAE, but not a Test Engineer.
And as I mentioned in my previous blog, part of the reason Test Engineers get little respect is because they are seen by developers as little more than testers. Although Test Driven Development has gained some traction, notice that it focuses on developers writing better unit tests. The concept of Development Driven Testing has unfortunately not caught on.
Monday, May 16, 2011
Test Engineers don't get any respect
Had I known the stigma of belonging to the Test group of a company, I probably never would have become a Test Engineer. Now technically, what I do isn't what most Test Engineers do, and although that is my official title, I am closer to what would be called a Software Development Engineer in Test.
What amazes me is the attitude people have in regards to the skill set of Test Engineers. While I am not a guru in any of these languages, I have written production code in C, C++, Java, C#, Perl, Python, SQL, and Labview. I have used (but again, am no master in) a broad array of technologies including SOAP (using Apache CXF), xmlrpc (using Apache ws-xmlrpc), Swing, SWT, some small Eclipse plugins, just to name a few. So although my knowledge isn't necessarily deep, it is broad.
When I first got to my new job, I was talking with one of the developers. I mentioned that at my previous job, I was a developer. He looked at me quizzically and said, "so you demoted yourself to a Test Engineer?". I had a puzzled look as well, because I had never been a Test Engineer, nor had I worked for a Test department before. Over time however, I grew to understand why such denigrating attitudes exist, and I will explore them in later blogs.
Nevertheless, I essentially had to prove that I could indeed program and even design relatively large scale projects by myself. Developers seem aghast when during code reviews I point out things that seem to surprise them. For example, code paths that they forgot to free a malloc'ed pointer to, or why you shouldn't write 1000 line functions (and god-forbid that they cut-and-paste a function from somewhere else). They also seem aghast when I counter their feeble argument that breaking down too many functions into sub-functions would cause overhead by telling them to A) profile it and B) inline their functions.
Still, there is some truth to the notion that many Test Engineers are not all that technically savvy, and so I don't blame developers for thinking that at least some Test Engineers are technically deficient. And this realization has made me determined to NOT be a Test Engineer again.
Just to relate one last story, I knew another Test Engineer who had to call someone at a company in order to purchase a product of theirs that would be utilized by their test framework. After some discussion, the sales rep basically asked to speak to the developers in charge, or someone who knew what this framework really did. The great irony was that the test framework was designed and programmed by the very Test Engineer the sales rep was talking to.
Test Engineers....we get no respect.
What amazes me is the attitude people have in regards to the skill set of Test Engineers. While I am not a guru in any of these languages, I have written production code in C, C++, Java, C#, Perl, Python, SQL, and Labview. I have used (but again, am no master in) a broad array of technologies including SOAP (using Apache CXF), xmlrpc (using Apache ws-xmlrpc), Swing, SWT, some small Eclipse plugins, just to name a few. So although my knowledge isn't necessarily deep, it is broad.
When I first got to my new job, I was talking with one of the developers. I mentioned that at my previous job, I was a developer. He looked at me quizzically and said, "so you demoted yourself to a Test Engineer?". I had a puzzled look as well, because I had never been a Test Engineer, nor had I worked for a Test department before. Over time however, I grew to understand why such denigrating attitudes exist, and I will explore them in later blogs.
Nevertheless, I essentially had to prove that I could indeed program and even design relatively large scale projects by myself. Developers seem aghast when during code reviews I point out things that seem to surprise them. For example, code paths that they forgot to free a malloc'ed pointer to, or why you shouldn't write 1000 line functions (and god-forbid that they cut-and-paste a function from somewhere else). They also seem aghast when I counter their feeble argument that breaking down too many functions into sub-functions would cause overhead by telling them to A) profile it and B) inline their functions.
Still, there is some truth to the notion that many Test Engineers are not all that technically savvy, and so I don't blame developers for thinking that at least some Test Engineers are technically deficient. And this realization has made me determined to NOT be a Test Engineer again.
Just to relate one last story, I knew another Test Engineer who had to call someone at a company in order to purchase a product of theirs that would be utilized by their test framework. After some discussion, the sales rep basically asked to speak to the developers in charge, or someone who knew what this framework really did. The great irony was that the test framework was designed and programmed by the very Test Engineer the sales rep was talking to.
Test Engineers....we get no respect.
Subscribe to:
Posts (Atom)