@michalorman.com

  • Integrating Auth0 with Vue and Vuex

    March 26, 2018

    While you can find very good documentation on Auth0 site for integrating Auth0 with Vue it’s based on managing state through propagating props and custom events. The more practical and real-world implementation would use Vuex. Therefore I’ve created an example project integrating Auth0 with Vue and Vuex. It’s based on a similar concept as original documentation but uses Vuex for state management and events handling.

    TL;DR

    Further in this post I’m describing some of the aspects of the integration. If you’re not interested though you can clone the example code from GitHub.

    Setup

    In this post I assume you’ve already setup an Auth0 account and created the Single Page Web Application client. If not refer to the documentation on how to do it. I also assume you’re familiar with OAuth2 authentication.

    For this tutorial you’d need to have npm with vue-cli installed on your machine with:

    $ npm i -g @vue/cli

    I assume you’re familiar with Vue CLI tool.

    Generate new Vue project with:

    $ vue create auth0-with-vue-and-vuex

    Out of the features dialog you need to pick manual option and select Router and Vuex to install necessary dependencies and configuration.

    Finally install the auth0.js library.

    $ npm i -S auth0-js

    Now we’re ready to go.

    Authentication service

    The usage of auth0.js is pretty straightforward. Create an instance of auth0.WebAuth providing your client configuration, request type and scope than use authorize method to start login process and parseHash to parse returned tokens. If you’re familiar with OAuth2 you’d know that tokens will be appended to the callback URL to which user is redirected to.

    As the Auth0 documentation suggests best way is to encapsulate that logic with reusable service. Let’s create one:

    // src/lib/Authenticator.js
    
    import auth0 from 'auth0-js'
    
    export default class Authenticator {
      constructor () {
        this.auth0 = new auth0.WebAuth({
          domain: '',
          clientID: '',
          redirectUri: 'http://localhost:8080/auth',
          audience: '',
          responseType: 'token id_token',
          scope: 'openid'
        })
      }
    
      login () {
        this.auth0.authorize()
      }
    }

    You need to provide domain, clientID, redirectUri and audience to your Auth0 client. Auth0 will generate that values for you which can be simply copy’n’pasted to the actual code. The login method initiates the authentication flow by redirecting user to the Auth0 login dialog.

    Creating user session store

    We need a place to instantiate our Authenticator and where to keep user session information. Since this post is about Vuex let’s go ahead and firstly create a store for user session:

    // src/store/modules/session.js
    
    import Authenticator from '@/lib/Authenticator'
    
    const auth = new Authenticator()
    
    const state = {}
    
    const actions = {
      login () {
        auth.login()
      }
    }
    
    export default {
      state,
      actions
    }

    If you created new project with vue-cli you probably have src/store.js file. You can use it as well but I prefer to rename that file to src/store/index.js and use modules for concern separation. Both solutions works fine and it’s the matter of preference which one to use.

    Our state is empty for now but we’d fill it up soon. Now we need to add this module to our Vuex store:

    // src/store/index.js
    
    import Vue from 'vue'
    import Vuex from 'vuex'
    
    import session from './modules/session'
    
    Vue.use(Vuex)
    
    export default new Vuex.Store({
      modules: {
        session
      }
    })

    Alright, now we’re ready to fire the authentication flow.

    Triggering authentication

    To initiate authentication we need to call login action from our store. Typically such action is fired in response to clicking some button or link usually in page top nav bar. Let’s create a Vue component for that:

    // src/components/Navbar.vue
    
    <template>
      <nav class="navbar navbar-dark bg-dark">
        <a href="#" class="navbar-brand">Auth0 with Vue and Vuex Example</a>
        <ul class="navbar-nav ml-auto">
          <li class="nav-item">
            <button class="btn btn-primary" @click='login()'>Sign In</button>
          </li>
        </ul>
      </nav>
    </template>
    
    <script>
    import { mapActions } from 'vuex'
    
    export default {
      name: 'Navbar',
    
      methods: mapActions(['login'])
    }
    </script>

    When clicking our login button the login method is called which dispatches the login action of our store. This will redirect user to Auth0 login dialog.

    Please note using bootstrap. Simplest way of adding it is pasting the CDN link in the index.html file. Alternatively npm package can be installed and imported in App component.

    We need to add our Navbar component the the App component:

    <template>
      <div id="app">
        <Navbar />
        <router-view/>
      </div>
    </template>
    
    <script>
    import Navbar from '@/components/Navbar'
    
    export default {
      components: { Navbar }
    }
    </script>

    Now we can initiate login flow. Go ahead and run npm run serve navigate to http://localhost:8080 and click the Sign In button. You should be redirected to Auth0 login page.

    Handling Redirect

    First part is done. We’re initiating authentication flow and redirecting user to login dialog managed by Auth0. Next step is to handle redirect callback. Auth0 would append auth_token and id_token to the redirect URL we’ve set in the initial request (http://localhost:8080/auth in our case). We should validate those tokens to ensure they’re generated by the trusted entity. Fortunatelly the WebAuth object we’ve instantiated in Authenticator has a build in method for that. So let’s start with extending our authentication service:

    export default class Authenticator {
      // ...
    
      handleAuthentication () {
        return new Promise((resolve, reject) => {
          this.auth0.parseHash((err, authResult) => {
            if (err) return reject(err)
    
            resolve(authResult)
          })
        })
      }
    }

    The parseHash from the auth0.js library would validate and parse tokens and expires_in which determines the duration of the access token. Since it provides the results in a callback the easiest way would be to return a promise so that it’s easier to use that in store action which we’re going to add now by extending our session store:

    import Authenticator from '@/lib/Authenticator'
    
    const auth = new Authenticator()
    
    const state = {
      authenticated: !!localStorage.getItem('access_token'),
      accessToken: localStorage.getItem('access_token'),
      idToken: localStorage.getItem('id_token'),
      expiresAt: localStorage.getItem('expires_at')
    }
    
    const getters = {
      authenticated (state) {
        return state.authenticated
      }
    }
    
    const mutations = {
      authenticated (state, authData) {
        state.authenticated = true
        state.accessToken = authData.accessToken
        state.idToken = authData.idToken
        state.expiresAt = authData.expiresIn * 1000 + new Date().getTime()
    
        localStorage.setItem('access_token', state.accessToken)
        localStorage.setItem('id_token', state.idToken)
        localStorage.setItem('expires_at', state.expiresAt)
      },
    
      logout (state) {
        state.authenticated = false
        state.accessToken = null
        state.idToken = false
    
        localStorage.removeItem('access_token')
        localStorage.removeItem('id_token')
        localStorage.removeItem('expires_at')
      }
    }
    
    const actions = {
      login () {
        auth.login()
      },
    
      logout ({ commit }) {
        commit('logout')
      },
    
      handleAuthentication ({ commit }) {
        auth.handleAuthentication().then(authResult => {
          commit('authenticated', authResult)
        }).catch(err => {
          console.log(err)
        })
      }
    }
    
    export default {
      state,
      getters,
      mutations,
      actions
    }

    That’s the full implementation of our session store. In that store we have login, logout and handleAuehtneication actions which will set through mutations the state of our session including storing it in local storage.

    Now we need to setup a route which would handle our callback redirect:

    export default new Router({
      mode: 'history',
      routes: [
        {
          path: '/',
          name: 'home',
          component: Home
        },
        {
          path: '/auth',
          name: 'auth',
          component: Auth
        }
      ]
    })

    Finally we need Auth component:

    // src/views/Auth.vue
    
    <template>
      <div class="authenticating">
        Authenticating...
      </div>
    </template>
    
    <script>
    import router from '@/router'
    import { mapActions } from 'vuex'
    
    export default {
      name: 'Auth',
    
      methods: mapActions(['handleAuthentication']),
    
      data () {
        this.handleAuthentication()
        router.push({ name: 'home' })
    
        return {}
      }
    }
    </script>

    The Auth component will handle user redirect and dispatch handleAuthentication action to validate and parse tokens. Finally it redirects to the /home path.

    Technically authentication is finished at this point but we have no UI indicator that tells whether user is authenticated and no way to destroy the session. Let’s update the Navbar component to address that:

    <template>
      <nav class="navbar navbar-dark bg-dark">
        <a href="#" class="navbar-brand">Auth0 with Vue and Vuex Example</a>
        <ul class="navbar-nav ml-auto">
          <li class="nav-item">
            <button v-if="!authenticated" class="btn btn-primary" @click='login()'>Sign In</button>
            <a v-if="authenticated" href="#" class='nav-link' @click='logout()'>Log Out</a>
          </li>
        </ul>
      </nav>
    </template>
    
    <script>
    import { mapActions, mapGetters } from 'vuex'
    
    export default {
      name: 'Header',
    
      methods: mapActions(['login', 'logout']),
    
      computed: mapGetters(['authenticated'])
    }
    </script>

    That is the final version of Navbar component which will render Sign In button and Log Out link when user is authenticated. Also would dispatch logout action to destroy user session.

    Go ahead and open http://localhost:8080. The authentication is fully functional at this point.

  • Blockchain by an eye of software developer

    February 20, 2018

    Blockchain without any doubt is recently the most frequently used buzzword. Hype is huge powered by the crazy world of cryptocurrencies. The craziness of cryptocurrencies is probably the reason why lot of software developers treat blockchain as something abstract and not interesting. Let’s have a look into a blockchain by an eye of a software developers.

    Debunking myths

    A lot of myths has raised and lot of developers has distorted image of blockchain. I won’t go over each myth those can be found easily on the web. I’ll just point out one and most important: Blockchain is not a Bitcoin or any other cryptocurrency.

    The common misunderstanding is the difference between blockchain and Bitcoin. Well blockchain is an underlying technology of Bitcoin but it’s not a Bitcoin nor a cryptocurrency. The cryptocurrencies are just features build on top of a blockchain. Because of that (and for few other reasons) blockchain is often being referred as distributed ledger to differentiate this technology from Bitcoin.

    So what is a blockchain? By an eye of a software developer it’s a data structure or a database or both. Well… it’s a set of records linked to each other and stored in a distributed repository. It’s hard to have a definition satisfying everybody as there are many variations of a blockchain and constantly new are being invented. To understand what blockchain is let’s have a closer look from a software developer perspective.

    Blockchain as a data structure

    Blockchain as the name stands is a chain of blocks. So what is a block than? Block is a chunk of data with accompanying meta information describing it. It can store pretty much anything from financial operations to binary code that can be executed - anything we want. Meta information validates the data stored in a block so that we can be assured the data was not altered. Let’s write a simplest script generating a block.

    require 'digest'
    
    version = 1
    data    = ARGV[0]
    
    stamp = format(
      "%d:%d:%s",
      version,
      Time.now.to_i,
      data
    )
    
    digest = Digest::SHA1.hexdigest(stamp)
    
    puts format("stamp: '%s'\ndigest: %s", stamp, digest)

    Let’s execute:

    $ ruby block.rb "Hello world"
    stamp: '1:1518860302:Hello world'
    digest: 52f77718efc94ab98fd7fc9ea0457a467e662af5

    We’ve got our data and a digest ensuring it’s integrity. As a data you can put literally anything from plain text, JSON to binary data like images, audio or video files to executable code. Anything you want.

    Right now nothing really exciting about the blockchain. Let’s proceed to the next element of a blockchain which is a chain.

    In blockchain each block is linked with the previous block. Each except the first block which is a special block called genesis block. Let’s modify our script allowing the hash of a previous block to be passed as a parameter.

    require 'digest'
    
    version       = 2
    data          = ARGV[0]
    previous_hash = ARGV[1]
    
    stamp = format(
      "%d:%d:%s:%s",
      version,
      Time.now.to_i,
      data,
      previous_hash
    )
    
    digest = Digest::SHA1.hexdigest(stamp)
    
    puts format("stamp: '%s'\ndigest: %s", stamp, digest)

    Let’s execute:

    $ ruby chain.rb "Genesis block" 0
    stamp: '2:1518860804:Genesis block:0'
    digest: 18567e288de0c22921929a77f3d98f23f91d376b
    
    $ ruby chain.rb "Hello world" 18567e288de0c22921929a77f3d98f23f91d376b
    stamp: '2:1518860817:Hello world:18567e288de0c22921929a77f3d98f23f91d376b'
    digest: c000f33b51a2c73869543e8b033bbb2ba6408afd

    We’ve created a genesis block and next we’ve added new block to a chain linked to a genesis block. But what exactly we’ve gained with that?

    The reason for this is to make altering the chain harder. If more blocks are added on top of a block I’ve added in order to change my block attacker must recalculate digest’s for my block and all blocks added on top of it since my block is a parameter for calculations of further blocks. If attacker wouldn’t recalculate all necessary blocks it can be quickly identified that chain was tampered.

    So if 1000 blocks were added on top of my block how hard for an attacker would be to change my block? Well let’s add another block and see:

    $ time ruby chain.rb "New block" c000f33b51a2c73869543e8b033bbb2ba6408afd
    stamp: '2:1518861367:New block:c000f33b51a2c73869543e8b033bbb2ba6408afd'
    digest: 1e09ba553668d19079d4ce5d718fc271464fca0e
    ruby chain.rb "New block" c000f33b51a2c73869543e8b033bbb2ba6408afd  0.08s user 0.04s
    system 48% cpu 0.235 total

    I’ve added new block on top of my previous block. It took 0.235s to calculate it. Hold on here for a minute. Recalculating 1000 blocks would take 235 seconds! With a simple ruby script! I’m sure motivated hacker can do way better than that. Where is that legendary blockchain security everybody is talking about?

    Proof of work

    Ok so we have a problem. We link next block with a previous block but we aren’t getting anything out of that. Calculating a hash is so fast that with any computer we can easily hack the whole chain. How to get it more secure then?

    So the problem is that we’re ensuring integrity of our chain with simple hash code that can be computed very fast. The fast computation is necessary as we need to be able to verify integrity in an instant. We can’t afford perform costly computations in order to verify data - that would be counter productive. If only we could make it hard to generate new block but make it effortless to verify it’s correctness. If only we could do something like that.

    Something like that already exists and is called Hashcash. It is a proof-of-work algorithm developed to limit email spam and denial of service attacks. The idea is very simple each email has to include special header for which hash was starting with certain amount of 0’s. Since you can’t predict the outcome of a hash function you need to randomly modify the header so that it’s hash matches the requirements. It’s hard to generate but easy to verify and it basically proves that computing power was spent when generating a block.

    Let’s modify our script so that it generates hashes starting with certain amount of 0’s.

    require 'digest'
    
    version       = 3
    data          = ARGV[0]
    previous_hash = ARGV[1]
    difficulty    = ARGV[2].to_i
    
    nonce         = 0
    
    begin
      stamp = format(
        "%d:%d:%s:%s:%d",
        version,
        Time.now.to_i,
        data,
        previous_hash,
        nonce += 1
      )
    
      digest = Digest::SHA1.hexdigest(stamp)
    
      print digest + "\r"
    end until digest.start_with?('0' * difficulty)
    
    puts format("stamp: '%s'\ndigest: %s", stamp, digest)

    In this script we regenerate blocks as long as it’s hash doesn’t start with given amount of zeros. We’re doing that by appending a nonce to the hashed data. We’re trying to guess the value for which hash is going to have specified amount of zeros. The more zeros at the beginning we expect the more difficult it’s to generate a valid block. Let’s have a look:

    $ time ruby hashcash.rb "Hello world" 18567e288de0c22921929a77f3d98f23f91d376b 4
    stamp: '3:1518906615:Hello world:18567e288de0c22921929a77f3d98f23f91d376b:36829'
    digest: 0000a8e2383121c83af44b03c2182f038c6de27f
    ruby hashcash.rb "Hello world" 18567e288de0c22921929a77f3d98f23f91d376b 4  0.39s user 0.09s system 79% cpu 0.611 total
    
    $ time ruby hashcash.rb "Hello world" 18567e288de0c22921929a77f3d98f23f91d376b 5
    stamp: '3:1518906629:Hello world:18567e288de0c22921929a77f3d98f23f91d376b:1445615'
    digest: 00000e0d26fbec9491624af44b70480e22913f8e
    ruby hashcash.rb "Hello world" 18567e288de0c22921929a77f3d98f23f91d376b 5  8.80s user 1.74s system 92% cpu 11.395 total
    
    $ time ruby hashcash.rb "Hello world" 18567e288de0c22921929a77f3d98f23f91d376b 6
    stamp: '3:1518906650:Hello world:18567e288de0c22921929a77f3d98f23f91d376b:1984015'
    digest: 000000d8edbddbc8f63b6cde64ebce3124a14e7e
    ruby hashcash.rb "Hello world" 18567e288de0c22921929a77f3d98f23f91d376b 6  12.40s user 2.47s system 94% cpu 15.783 total

    Hash with 4 zeros was generated in 0.6 second, with 5 zeros in 11 seconds with 6 in 15. By setting the amount of required zeros we can adjust the frequency of new blocks being added to the chain For example Bitcoin adjusts the difficulty in a way that new block is added in average every 10 minutes.

    Mining

    In real world block needs to have a hash starting with way more than 4 or 5 zeroes. At the moment of writing this post Bitcoin block starts with 18 zeros. You need a substantial computing power to generate such block within a reasonable time. That is actually a reason why blockchains are so hard to modify. If hacker wants to modify a block he’d need to modify all blocks on top of it (and convince other nodes that his version of a chain is actual). It would take a lot of time and resources to do so for a single person and for sure new blocks will be added to the chain while he’ll be recalculating his version.

    Therefore the concept of miner was introduced to the blockchain. Basically miners are volounters offering their computing power in order to find a proper nonce giving a block with desired hash. There may be different conditions under which block is valid, hash starting with zeros is just one example (and popular one) but it could be anything. Miners are receiving gratification for their effort. In cryptocurrencies world it’s usually some fee assigned with a transaction but it can be anything. Miners can be paid with real money as well.

    The network of miners collectively are providing such computing power that within a blockchain network new blocks can be mined in a reasonable time. By adjusting a difficulty it’s possible to control how often new blocks are added to the chain.

    Distributed Ledger

    Another aspect of blockchain security is distribution. By distribution we understand that there is no one central authority. Copies of a blockchain are distributed across all nodes within a network. That introduces a few problems.

    When adding a new block to the chain, nodes need to agree that particular block is valid and will be added as a next block. This problem is called a consensus problem and there are few strategies how to solve it. Usually there is some threshold that if for example more than 50% of nodes agree that block is valid it is added to the chain and new version of chain is distributed across a network.

    Consensus algorithms are a broad topic and I’d look onto them in future posts.

    Conclusion

    In this post I’ve only touched the surface of a blockchain topic. The main take away should be that for a software developer blockchain is like a data structure or repository. It has nothing to do really with cryptocurrencies as most people think. It is a way of storing a data or records in a distributed network with a few clever tricks making it more secure than ordinary ways of storing data. It can be brought even further as stored data may be something executable allowing software developers to develop decentralized, distributed applications.

    Understanding a very basics of a blockchain should put more light on what it is and how this technology can be utilized (and that it has nothing to do with speculation on BTC).

  • Base64 And String Encoding

    June 16, 2017

    TL;DR

    Always explicitly set encoding for Base64 decoded input as it defaults to ASCII.

    Base64.decode64(input).force_encoding('utf-8')

    Base64 encoding problem

    Base64 comes handy when we need to represent binary data in ASCII. For example when transferring a file or storing encryption key.

    The other use case when Base64 is useful is when we can handle only ASCII characters but user may provide non-ascii strings. In such case we encode string on one end transfer encoded input and decode on the other end. You may however be surprised that this may not work straight in Ruby.

    encoded = Base64.encode64('Zażółć gęślą jaźń.')
    => "WmHFvMOzxYLEhyBnxJnFm2zEhSBqYcW6xYQu\n"
    Base64.decode64(encoded)
    => "Za\xC5\xBC\xC3\xB3\xC5\x82\xC4\x87 g\xC4\x99\xC5\x9Bl\xC4\x85 ja\xC5\xBA\xC5\x84."

    Decoding what was previously encoded with standard Base64 class doesn’t return the same value which was encoded. It is because Base64 can’t determine input’s encoding and decodes in 8-bit ASCII:

    Base64.decode64(encoded).encoding
    => #<Encoding:ASCII-8BIT>

    It is very difficult task to guess input’s encoding but surprisingly decode64 doesn’t accept any parameter allowing to set the desired encoding and just defaults to ASCII.

    To make above example work string encoding needs to be forced manually:

    Base64.decode64(encoded).force_encoding('utf-8')
    => "Zażółć gęślą jaźń."
  • Cute Trick With Pdfinfo

    December 04, 2016

    With the pdfinfo tool you can easily check the size of every single PDF document page by providing the -l switch:

    $ pdfinfo -l 5 demo.pdf
    ...
    Page    1 size: 592.56 x 792 pts
    Page    1 rot:  0
    Page    2 size: 592.56 x 792 pts
    Page    2 rot:  270
    Page    3 size: 592.56 x 792 pts
    Page    3 rot:  0
    Page    4 size: 592.56 x 792 pts
    Page    4 rot:  270
    Page    5 size: 592.56 x 792 pts
    Page    5 rot:  0
    ...

    The problem is that you need to know the document pages count as a -l switch accepts the number of last page to examine. That would require to execute pdfinfo twice: first to get the pages count, second to get page sizes. However there’s a trick to do everything in single call providing -1 as the last page number:

    $ pdfinfo -l -1 demo.pdf
    ...
    Page    1 size: 592.56 x 792 pts
    Page    1 rot:  0
    Page    2 size: 592.56 x 792 pts
    Page    2 rot:  270
    Page    3 size: 592.56 x 792 pts
    Page    3 rot:  0
    Page    4 size: 592.56 x 792 pts
    Page    4 rot:  270
    Page    5 size: 592.56 x 792 pts
    Page    5 rot:  0
    ...

    I didn’t found any official documentation specifying that just discovered it by trial and error.

  • 8 Lost Skills With Rails

    June 29, 2015

    Rails is great because it’s so easy. Rails is awful because it’s so easy. The same thing which makes that framework so good for developers, especially for beginners, makes it so bad for real life programming. The point is that a framework is not all. Tens of years of software development didn’t came up with just learning a framework to develop great products. Lot of decisions which Rails made for us is fine in most cases, but makes lot of developers unaware that there is more than just a framework their are using now and eventually they will work with another stack.

    Below I’m listing skills that got lost or developers aren’t paying much attention mastering those. To be honest not all were lost due to the Rails itself, some are related to Ruby language and ecosystem.

    Logging

    Watching Rails logs makes me cry blood. There is a reason why events should be logged in just one line. It is because of the parsability. Repeat it: logs should be optimized for parsing not reading. Well to be exact, readability is important in development, but production logs should always be easily parsable.

    I saw developers that care about indentation and line wrapping when logging some JSON/XML responses. I was wondering why they didn’t add syntax highlighting.

    Logs format isn’t even the biggest problem, as changing it is quite easy (eg. with this gem). The biggest problem is that developers don’t know when and what to log. Many times I saw logging of completely meaningless information whereas important things like 3rd party requests, parameters, response (codes / message) where completely ignored. Logging all those events is invaluable when debugging production running applicationa and in any post-mortem crash analysis. In most cases however, I’m missing those information in app logs and need to rely on my intuition.

    Obiect Oriented Programming

    Ruby is object oriented language. It’s a pitty that so few developers really spent time learning principles of object oriented programming. Developers don’t care about proper abstractions, encapsulation, coupling. They just hack out the solution. In most cases the excuse is that the code is doing what it should.

    Good code not only does what it should. Good code is readable, testable, maintainable and extendable. Note that I’ve put readability first. Code is read more often that it is written. Source code serves as one and only documentation (comments or word documents lie!).

    There is a trend in Rails and Ruby to develop with the most sophisticated techniques developer can use. Convenience is used before the readability. I believe it should be the opposite. Convenience is good whey you’re writing a framework or a library. In real life coding readability is more important as at some point you or someone else will come back to things that have been implemented and would need to refactor it.

    Lately we see huge conversion towards good object oriented programming. We see programmers implementing lightweight, single-purpose, stateless services. That’s good, but single responsibility is not the only SOLID principle. Now it’s time to learn Rails developers to write code that is open for extension, but closed for modification.

    SQL and DB modeling

    ORM did lot of good for software development, and about the same amount of damage. The damage is increased by the solutions like Rails migrations where database schema is build automagically with bunch of some method calls and rake tasks.

    So whats the problem if ORM and migrations are so convenient? The problem is the convenience and laziness of software developers which do not verify that generated schema is actually correct (or efficient). In most cases, especially in Rails schema is missing indices. Indices are missing for obvious places like foreign keys, but also for columns which are searched or ordered by, though those are less obvious places.

    Similar thing with querying the database. Developers do not verify the SQL queries leaving the ones generated by the framework. In most cases it is fine, but there are so typical cases where developer should verify the query for things like n+1 problem. So much time I see typical places which screams for includes statement.

    Problem with database modeling is especially visible when using NoSQL databases, where developers tend to put lot of garbage and just do not care. At some point however this mess hits back and hits very hard.

    Design and Architecture

    Every Rails developer knows what MVC acronym means. Less will know the origins and what was the original purpose of this pattern. Even less will know anything other than MVC. With DHH advocating for removing layers and indirections the Rails community came with flat applications, which are fine for small to mid size. For anything more complex it isn’t enough though. Flat architecture Rails is promoting just doesn’t work well for complex apps. We end with god classes, callback hell, duplicated logic and complicated tests (if any as such cluttered code is untestable in most cases).

    The point is, that framework should be an implementation detail, not the application itself. In Rails it is the opposite. It works well at the beginning, but in complex scenarios you need to fight against the decisions that framework developers made for you. Despite what DHH is saying adding layers and indirections is not bad idea. It’s neither bad nor good. It’s contextual and developers should decide given the context of their application.

    Rails developers tend to not see any other way of implementing app that they were teached while learning the framework. When you talk about design patterns or DDD they will call you some Java zealot. However the Rails Way is only one of many approaches and the problem should determine the solution not the opposite.

    Testing

    That point may be controversial, as everybody nowadays is TDD-ing, BDD-ing, CI-ing. The problem is that despite code has tests it is poorly tested. Tests, if testing anything at all, completely skip testing edge/corner cases. Those are the real evaluation whether the piece of code is working correctly or not. In most cases tests are focused to sanity check some success paths and nothing else.

    This problem especially escalated after DHH was advocating against the unit testing in respect of integration tests with full stack involved. Integraion tests are time consuming and requires lot of code to be written in order to be able to execute, therefore developers start testing in more general levels and in least scenarios. Also when you’re doing integration test it is harder to simulate edge case than it is in unit test.

    Another problem with tests is that developers do not listen to them (even when TDD-ing). The setup pain screams for refactor, but developers tends to leave the code as is because it is doing what it should. Lot of time I see developers do not even start with red test not understanding why it is important to ensure that test is failing for a right reason.

    I/O

    The most typical issue with I/O I’m encountering is HTTP file downloading. For some reason developers tend to download the whole file, read its whole body into memory and than save it as an output files. All of that without any reflection. Developers tend to not care about the memory consumption. Streams and pipes are complete abstraction and so low level that not even worth thinking about.

    There are even libraries to popular cloud storage services which provide no other way of downloading a file than to read its whole content into the memory, unless one implement raw client to that service.

    With all the high-level abstractions over the I/O developers tend to not understand nor care what is happening at low level. The concepts of streams, pipes or non-blocking I/O seems to be complete magic, not even speaking about system processes, threading or concurrency (which probably should be a 9th lost skill in this post). The virtualization made developer careless about the resources, but not always you have the option to deploy on Heroku or Amazon or infinitely scale-out. Virtualization will help in most cases but there will be few where careless resource management will hit back.

    UNIX Shell

    Ruby is a scripting language that works well with the UNIX shell. Developers tend to forgot about that. Also lot of things are solved by the shell tools yet developers tend to be reinventing the wheel because their coding Ruby!

    For some reason developers don’t know how to embrace the power of UNIX shell. Commands are badly executed, developers don’t care whether messages are logged to STDIN or STDERR yet lot of important debug information is just thrown away. Commands are assumed to be run successful despite of the returned status code.

    UNIX shell is developed for more years then most of developers live. Lot of common tasks can be solved just by using what shell is providing. Learning the shell should be one of the most important skills to acquire, but it is not as impressive as showcasing asynchronous bidirectional JavaScript communication.

    It is a common belief that shell is for devops. I believe this belief is not true.

    Algorithmics

    Last but not least, though maybe least important from the above set of skills. It is also least related to Rails, more to the richness of Ruby standard library and gems.

    The thing is that developers given with all typical algorithmic problems already solved and provided as easy to use plugins, do not know or care about how they are solved. Also they find difficult solving certain class of problems in an efficient way (efficient from the CPU and memory usage point of view). They just hack the solution with bunch of if's and while's. I know, I know. Currently we have wells of RAM and almost unlimited CPU power, especially in web development due to the virtualization, yet there are cases (like mobile devices) where resources matter (and battery consumption).

    I’m not saying that developers should implement all the lists or sorting on they own, but they should stop claiming that it is not their concern which should be solved by the platform / framework they are using. From time to time they will find a problem which they will need to solve with kind of algorithm or data structure, or maybe some implementation will misbehave in given context and they should be able to identify such case and know the alternate implementations.

    Currently developers tend to don’t care at all in learning algorithms and data structures. It sounds like something for long-beard geeks.

    Conclusion

    I understand that nowadays the code must be developed rapidly. Thats how the state of development is in current market. But rapid development combined with frameworks embracing magic and undereducated developers is a ticking bomb. The urgent need for software developers forced us to cut corners in their education. Problem is that those education gaps aren’t filled over time. Frameworks like Rails, that abstracted away lot of concerns, are helping novice developers in being productive quickly, but for a cost of damaging their knowledge. Every Rails developer should be aware of that and gradually learn things he weren’t told in his Rails course.

    Software development is not just solving all problems with the same hammer developer was given. Improvement is not only learning new frameworks. At some point every developer should learn the raw basics. It’s even more important as that basics spans across every platform or framework - just implemented using different keywords.

  • Redirect STDERR to STDOUT

    March 24, 2015

    If you ever executed a shell command from a Ruby code like this:

    result = `identify photo.jpg` # => photo.jpg JPEG 128x128 128x128+0+0 8-bit DirectClass 1.51KB 0.000u 0:00.000
    raise Error unless $?.success?

    Well, you’re doing it wrong! Okay, okay. It will work but you are hurting yourself (or perhaps any developer that will maintain such code later).

    Most of well written shell commands provides useful information whenever something goes wrong. For example:

    $ identify foo.pdf
    identify.im6: unable to open image `foo.pdf': No such file or directory @ error/blob.c/OpenBlob/2638.
    $ identify sample.doc
    identify.im6: no decode delegate for this image format `sample.doc' @ error/constitute.c/ReadImage/544.

    You can immediately tell what’s the problem when executing those commands. You may be thinking that this message will be stored in a result variable from the example above, but this is not true:

    $ identify sample.doc 2>/dev/null
    $

    No output at all! The message instead of standard output was written to standard error. Executing shell command using backticks in Ruby returns whatever was written into a STDOUT thus all the helpful error messages are gone. That leads you to hard to find problems.

    You can easily redirect standard error into a standard output in bash by appending 2>&1 to the executed command. So proper way of wrapping bash commands in Ruby should be:

    result = `identify photo.jpg 2>&1`
    raise Error, result unless $?.success?

    That way the exception raised will contain the error message which command put into to the STDERR. So make yourself (and other developers) a favor and as a rule of thumb redirect STDERR to STDOUT in wrapped shell commands.

    TL;DR

    It is even more important when the command you’ve executing seems to be ignoring UNIX practices and returns always 0 as a exist status code. That happened to me lately and because of lack of redirection the command was always assumed to be executing successfully (despite of the fact that device was unplugged and tool was logging errors to STDERR).

  • Daemons In Rails Environment

    March 19, 2015

    Suppose you need to write a system daemon running in the context of Rails application, so that you have access to your models or services. You might be tempted to use gems like daemons but using such gems you’d need to load Rails environment manually as those are not aware of Rails application context. While such gems provides advanced functionality - like monitoring - they are additional dependency that must be maintained. There is a simpler approach though that will work fine in most cases.

    The easiest way to load Rails environment is to execute rake environment task. Combining it with any other rake task we can easily implement a daemon running in the context of Rails application. In fact Resque (in version 1.x) embraces this approach. By reading its code you can came up with the following template for simple daemons running in the context of Rails:

    task start: :environment do
      Rails.logger       = Logger.new(Rails.root.join('log', 'daemon.log'))
      Rails.logger.level = Logger.const_get((ENV['LOG_LEVEL'] || 'info').upcase)
    
      if ENV['BACKGROUND']
        Process.daemon(true, true)
      end
    
      if ENV['PIDFILE']
        File.open(ENV['PIDFILE'], 'w') { |f| f << Process.pid }
      end
    
      Signal.trap('TERM') { abort }
    
      Rails.logger.info "Start daemon..."
    
      loop do
        # Daemon code goes here...
    
        sleep ENV['INTERVAL'] || 1
      end
    end

    Above daemon may work in background as well as in foreground. We can specify the pid file, interval and log level. It can be started with following command:

    BACKGROUND=y PIDFILE=daemon.pid LOG_LEVEL=info bundle exec rake start

    To kill daemon:

    kill `cat deamon.pid`

    Monitoring should be delegated to tools like Monit which are far more powerful than any gem providing daemon functionality. Finally we are’t introducing any additional dependencies which is always a good thing.

  • ← Older Newer →