Sharding /dev/null - Latest Comments

Re: Sharding /dev/null

ZoIomon — Wed, 29 May 2013 06:53:00 -0000

Now I want to learn Outlaw Techno Psychobitch!

Re: Sharding /dev/null

gar1t — Tue, 16 Apr 2013 11:52:54 -0000

These are from Landslide. Alas I have the same problem viewing them on my Android phone :( I'd love some recommendations on a slideshow generator that creates universally readable HTML! The idea of using Landslide in the first place was that it made accessing these slides easy for people.

#fail

Re: Sharding /dev/null

Le Viet Bach — Tue, 16 Apr 2013 07:52:21 -0000

Which library did you used to create these slides? Btw, please consider adding on screen buttons if possible. The slides can't be read from a tablet

Re: Sharding /dev/null

Paul Barry — Fri, 05 Apr 2013 04:23:16 -0000

Having watched the "new" video, I think I'm going to need some serious therapy... or a stiff drink.

Re: Sharding /dev/null

gar1t — Wed, 27 Mar 2013 11:27:06 -0000

Re: Sharding /dev/null

Joe — Wed, 27 Mar 2013 10:52:40 -0000

"Jolly Good" indeed.

Forced Pair Programming will never be the same.

Re: Sharding /dev/null

Joe Armstrong — Mon, 25 Mar 2013 01:27:26 -0000

Great - Hollywood next stop - we thought "OTP" was a "Jolly Good" name with wide appeal at the time

Re: Sharding /dev/null

Robert Virding — Fri, 22 Mar 2013 10:04:20 -0000

No worries Garret. We truly loved it!

Re: Sharding /dev/null

Kotedo — Fri, 22 Mar 2013 03:00:01 -0000

Well done!

Re: Sharding /dev/null

gar1t — Sun, 03 Feb 2013 17:15:28 -0000

It's impossible to argue that tests are *bad* and should be avoided. The One True Religion doesn't say that. It says that it's acceptable to trade time writing tests for time making your code *obvious*.

If your code is obvious and you have nothing better to do, write tests.

All of the advantages of tests that you list are True and Good. But they cost time and effort. If that time and effort is well spent, write tests.

If your code is sensitive to regressions, write tests.

But what's the point of writing tests for incomprehensible code? I've frequently seen code put into production that *sucks ass* only because it appeared thoroughly tested. The tests indeed passed -- but the whole ball of twine was so impenetrable no one could reason about errors when they occurred.

Incidentally, the bug in the original code example would have been caught the first time the code was *run*. That's one of the virtues of Erlang in particular -- you get a high degree of runtime testing from pattern matching.

Re: Sharding /dev/null

indrayam — Sun, 20 Jan 2013 15:59:14 -0000

I am assuming there are no videos of this event that I can peruse.

Re: Sharding /dev/null

Nick Vaidyanathan — Sun, 18 Nov 2012 20:45:29 -0000

This is a fantastic post. This code is beautiful...almost perfect. There is one small flaw, my son. You have too many functions at the same level of abstraction.

"Wat? That's the whole fucking point!" You may say.

Not so fast.

def print_db_table_row_counts(db_name): cur = db_cursor(connect_db(db_name)) print_row_counts(row_counts(all_tables(cur), cur))

Is at a higher level of abstraction. It asks for two different pieces:

a database manipulator that works with the db
a presenter that emits output.

You are missing classes. If you hate the class keyword, fine, call it a module. In the end, the point of encapsulation and separation of concerns is not only to open the way for modification/extensibility, it is also to manage cognitive load.

The classes, even if they end up being static classes that store no state, serve as a logical boundary of related functionality. They keep like concerns together and make it easy to see the actors in the play of your application.

Clearly you see how

def connect_db(db_name):
def db_cursor(db):
def all_tables(cur):
def row_counts(tables, cur):
def row_count(table, cur):
def db_name():

Belong together. It is not that much more work. It does not hide our meaning, but rather enhances it, when we place these in DatabaseManager.

We could self-flagellate, of course, and make a DatabaseConnector, TableQuerier, and RowCounter...but clearly this seems too fine a granularity, adding almost no additional value.

In the same way that you have correctly derived that functions should follow Miller's Magic Number to reduce complexity (http://en.wikipedia.org/wiki/T... , so too should your classes. This metric scales.

7+-2 lines in a function, functions in a class, classes in a package, packages in a library, libraries in a framework, frameworks in a component, components in an application, applications (like ls | grep) in a higher order application (e.g. enterprise application).

"In the garden man and woman were tricked by the ORM, which was disguised as a serpent. The ORM promises ease of use and productivity, but poisons applications with deformed, grotesque, incomprehensible SQL."
Yea, My Son. But Ye Must Experience The Glory of Proper ORM in GORM and dynamic finders, the way we were all searching for. http://grails.org/doc/latest/g...

Re: Sharding /dev/null

Nick Vaidyanathan — Sun, 18 Nov 2012 20:21:02 -0000

I was with you until the part about tests, where you betrayed our One True Religion:

"In our One True Religion, it’s no sin to trade key strokes, which are cheap, for clarity, which is precious."
"Our religion emphasizes clarity over economy."

It's just as trivial and simple to write tests that sanity check that functions that do one thing as doing the right thing. That they properly sanitize user input and set good boundaries, that they correctly call the next function with the proper set of arguments, and that if any logic, transformation/modification, is done by that function, it is done correctly.

If you would've had a unit test that compared expected versus actual outputs for your stacktrace bug, you would've caught the bug in 2 minutes instead of relying on readers to find it.

You are on the right track with this code. Engineers interested and professional enough to read and contemplate this MUST READ Robert Martin's Clean Code. But it is simply ridiculous and unrelated to argue that doing this obviates the need for tests.

Tests
- Make sure Ass O'Cluckolus, who looks at the system and modifies a feature after you, doesn't fuck up the existing logic and blindly copypasta into error.

- explicitly create a usage API. Wonder how it works in practice? Look at the integration tests.

- document requirements by creating an executable verification and validation mechanism. On every check-in, a continuous integration server can verify that no regressions have been introduced by running the tests.

- help drive design by forcing you to think about the problem, potential inputs/outputs and expected state, at every level of granularity.

- are cheaper. 10 minutes of debugging a unit test by looking at the 8 effected lines of code >>> 2 hours trying to trace execution flow through a chained A->B->C->D->E mess only to discover that C was making the input 3 instead of 4.

- Help drive conversations with your product manager. If they want some "soft, cuddly" requirement that says "should be fun to use", you can make that actionable by saying "let's define a scenario where fun to use = true, and one where fun=false."
Instead of storing that in Tribal Knowledge or static documentation, we store that AS CODE that has to change when our understanding of the system's performance envelope changes.

- Help solve arguments about design and implementation quickly. Instead of arguing with your peers for hours/weeks in meetings/discussions about whether something "may work",or whether something would be "dog slow", you write tests that mock simple scaffolding and quickly verify whether an approach is valid. Test written against interfaces form the Tracer Bullets (The Pragmatic Programmer)/architectural prototype that drives system.

If you're not doing Behavior Driven Development, putting the user desires as a result of Acceptance Test Driven Development in comprehensible and verifiable assertions of the behavior your software is expected to exhibit in various contextual conditions...YOAR DOING IT WRONG. These tests are small, they're easy, and they need to exist as run-time documentation/specification, design guide, and guard against unintended regression.

Re: Sharding /dev/null

Rob — Thu, 15 Nov 2012 00:42:42 -0000

Right on ! I feel the same way about Microsoft -- except for one little thing --
It isn't that they don't innovate anymore -- I can't think of a time that they EVER innovated anything significant. They started by buying DOS from a small developer in ~1981. Windows 1.0 was an obvious rip-off of the Mac, and they were forced to change it. They took the spreadsheet from Lotus123. The original MS Word (which was very good for it's time) -- was also bought from a small developer.

Re: Sharding /dev/null

Wmealing — Wed, 07 Nov 2012 07:58:34 -0000

Any ideas when the video will end up being uploaded. Keen to see it happening.

Re: Sharding /dev/null

Anonymous Coward — Wed, 24 Oct 2012 03:41:17 -0000

Hey, your RSS/Atom is 404ing. Please fix so I don't miss your awesome videos!

Re: Sharding /dev/null

Fred Alger — Sun, 14 Oct 2012 08:36:38 -0000

I'm not an Erlanger but I love the focus on simplicity. It reminded me of my pet favorite language, Factor, so I wrote a little post about it: https://gist.github.com/388840...

Re: Sharding /dev/null

Jan Willem — Sun, 05 Aug 2012 16:07:29 -0000

Really well written with a focus on the essence of Erlang which really invites a programmer to write beautiful code and split it up into small pieces that are obviously correct. It takes some time to get used to single assignment variables and pattern matching. But those are simple to grasp. And once understood the rewards are immense.

Re: Sharding /dev/null

Deanwagner13 — Sat, 04 Aug 2012 10:42:59 -0000

Great work. Just spent weeks with yaws, chicago boss, mochiweb and webmachine. I love webmachine, but I have a need to provide quick interfaces to Erlang code via http. This is nice.

Re: Sharding /dev/null

gar1t — Tue, 03 Jul 2012 07:45:10 -0000

Yes - I meant Prolog -- and Lisp, not LISP :) Corrected. Thanks!

Re: Sharding /dev/null

thanosqr — Tue, 03 Jul 2012 07:13:47 -0000

Awesome article; I am heading to refactor some of my Erlang code at the moment.
When I came to Erlang from Prolog I felt I lost a bit of declarativeness; but I think that this will change. Something I noticed: when you mention Prologue, did you mean Prolog perhaps?

Re: Sharding /dev/null

lambdadev — Sat, 30 Jun 2012 05:53:27 -0000

Great article, really. I think you're ready for lisp/clojure now ;)

While coding Erlang I don't like function imbrication, I prefer some more "monadic" style.
I call a function, with the result of that function I call another one, etc etc...
It's less clear because you have a foldl but you win another point for clarity : you see functions in order (not the other way around à la Lisp)
Funs_to_apply =
[fun create_db_args/1,
fun (Db_args) -> create_db(DB_args, Msg, State),
fun handle_create_db_result/1)],
lists:foldl(fun(Fun, Acc) -> Fun(Acc) end, Msg, Funs_to_apply).

It works like this : You have some data, call this function, then this other function, then is other one..

Of course I can be clearer with a function wrapping that foldl .

What bothers me is the lack of tests. I understand that some functions don't need to be tested because in a way :- they are simple translations
- Or they simply some clarification, you symbolize what is happening (and thus don't really need to test tit)

But to me, in Erlang more than in other languages, you do need tests because of the lack of typing. I mean, I need to write somewhere that the result of my function is Either {ok, this} or {error, that}.
Or that my result is a list of four values. etc.
Smaller functions make the test code simpler too. In fact you copy/past what you do in the REPL in a test and that's it, end of the story. That way, you have a test that explain what your function do. And the test is so short you have no issue in throwing it

But there is another problem that I can't (for the moment) really easily put words on it. When you have many functions, many functions calling one the other, at a point it's harder to test because functions are doing a huge tree of function calls. If you test one entry point you don't know at what level of function calls it breaks. The more I write tests, the more I try to have simple functions and you call one, then another, then anotheretc etc.. Chaining function call, not nesting one in the other

Re: Sharding /dev/null

gar1t — Wed, 27 Jun 2012 09:19:11 -0000

Thanks Ulf! I'm always fascinated to hear Erlang war stories.

While I've only started to adopt this style consistently, it's becoming clear that embarrassing obviousness is not just useful for correctness -- it brings sanity to change management.

When each piece of the puzzle is small (as you say, solve only one type of problem), with well reasoned inputs and outputs, changes tend to have limited impact. I can say, "here's where this fits -- it goes right into this slot" and modify the code with minimal ripple effect. It's the same refactoring process, but applied to new requirements!

Re: Sharding /dev/null

Ulf Wiger — Wed, 27 Jun 2012 08:15:05 -0000

Great post! And the presence of the bug in the first refactoring even made it better. ;-)

The thing illustrated by the bug is of course that functions, in addition to being short and clear, should solve only one type of problem (just as your create_db() function should only worry about creating the db, not dealing with messaging issues). Another important aspect is that if you can break the code down into short functions free of side-effects, these functions are very easy to test.

Either they are so embarrassingly obvious, as you say, that you don't really need to test them, or they are so easy to test that you can do so anyway. And once you have tested the function for all legal inputs (which you can, if the function is sufficiently limited in its scope), it will stay correct, forever, until requirements change.

This is something that we observed even in very complex erlang-based products: a large part of the code, once battle-tested and corrected, would tend to stay correct, even across major feature releases. Indeed, for each major release, we would tend to have lower bug density, even if we added lots of features. In some code, we never saw a single bug from the field! (for years, at least - I don't know what happened after I left) In many cases, that was code tested with QuickCheck.

That is where this type of refactoring work really pays off.

Re: Sharding /dev/null

bossek — Mon, 18 Jun 2012 09:42:10 -0000

The Code is much happier now :-)
Na zdraví!