COMPUTER SCIENCE, TECHNOLOGY

Will the real RESTful API please stand up

First things first – I am interviewing engineers for Pengala for the fast growing engineering team. This is the job link, so please apply if you are interested.

I’ve interviewed about 60-70 people so far,but during the all of the interviews(yes all of them) no one could really define RESTful API’s correctly. I realized that developers have a very convoluted understanding of what RESTful API’s are and in fact many of them use these API’s blindly. I know a lot has been written about it but most of it is either very large documentation or just shows best practices.

So I want to debunk some myths about what RESTful API’s are.

  1. REST is a protocol like HTTP ==> Incorrect, REST != HTTP
    1. Let me explain in bullets
      •  REST stands for Representational State Transfer. It is a framework – this can get confusing again, what the hell is a framework. So think of REST as a SET OF CONSTRAINTS(we’ll get to what those constraints are) to build highly scalable fault tolerant client server systems. The world wide web is actually a great example(and the biggest, but again slightly flawed) of REST.
      • It IS NOT A PROTOCOL. A protocol in web parlance is a communication interface or rules that define how components communicate on the interweb (from types of data to the interface types)
      • HTTP is a protocol. Hence REST ! = HTTP. But its the most used transport protocol with REST. You could as well use FTP or SMTP, but nobody likes those.
  2. But most probably in your case and in most cases HTTP API == REST API, i.e. almost certainly your REST API’s aren’t pure REST, hence you really cant call them those
    1. Let me explain again in bullets
      • Let me drill down the idea again – REST is a set of constraints. What are those constraints
        1. CLIENT – SERVER – Quite obvious
        2. Stateless – Every request you make must contain state information, this is the clients responsibility and the server is stateless
        3. Cache – Response messages from the service to its consumers are explicitly labeled as cacheable or non-cacheable.This way, the service, the consumer, or one of the intermediary middleware components can cache the response for reuse in later requests.
        4. Layered architecture – A REST-based solution can be comprised of multiple architectural layers, and no one layer can “see past” the next. Layers can be added, removed, modified, or reordered in response to how the solution needs to evolve.
        5. Code on Demand(optional) -This optional constraint is primarily intended to allow logic within clients (such as Web browsers) to be updated independently from server-side logic. Code-On-Demand typically relies on the use of Web-based technologies, such as Web browser plug-ins, applets, or client-side scripting languages (i.e. JavaScript).
        6. Uniform contract – Probably the most important of the lot and defines the REST aspect of REST API’s.
      • Lets look at Uniform contract and what that means.The diagram essentially shows the important concepts. 
      • Lets now focus our attention on the word REST shall we. Representational State transfer. So clearly there It’s about transferring representations of the state. Of what though?  Of Resources. Now what the hell are resources. A resource is a contextual mapping of entities that can fit into the URL model of the client – confused yet ? Example . Lets say I want to view a bank account from a database table on a remote server. My resource entity here is an account. Now if I want to get the detail of my account, how do I get it? I make a call over HTTP like this  => serverurl/v1/account/acc_number which inturn is doing this
        • GET /account/acc_number HTTP/1.1
          Accept: application/json
        • The resource here is account, the Accept:application/json portion is basically the representation of data that is being transferred. And we are doing this using the HTTPVerb GET.
      • So most important concepts here are Resources, Representations(Media Types) and HTTPMethods
      • Now REST Calls for HATEOAS – a really weird acronym that stands for Hypertext As The Engine Of Application State. This basically means that a main constraint of any restful api is that the responses drive the API usage directions. i.e. because the server is stateless and the client has to know what the endpoints are to call , this data must be transferred back in the response as hypertext i.e. URL’s. Confused. Lets take the previous example.
          • GET /account/12345 HTTP/1.1
            Accept: application/json
          • Now with HATEOAS the expected response would be something like this(scroll sidewise to see the response)
            HTTP/1.1 200 OK                                          <?xml version="1.0"?> <account>  <account_number>12345</account_number> <balance currency="usd">100.00</balance>   < link rel="deposit" href="/account/12345/deposit" />    <link rel="withdraw" href="/account/12345/withdraw" />   <link rel="transfer" href="/account/12345/transfer" />   <link rel="close" href="/account/12345/close" /> </account>
            
          • As you can see, the reponse, the “link”-tags allows us to find out the URLs that are needed for the specified actions for say Deposit or withdraw. THIS INFORMATION YOU WOULD HAVE NO CLUE ABOUT AT THE BEGINNING unlike most “REST API’s” today which document these. Getting the point?
      • So for a RESTful API to be truly RESTful it has to have the uniform structure of Resources, Representation and Methods as well as conform to HATEOAS.
      • IN GIST – HATEOAS basically implies that the only thing the client knows is the URI endpoint and everything from then on has to be driven by hypertext responses. This is violated the moment you document and expose individual. Another example is, lets say I am on STACKOVERFLOW. The only thing I know is the URL, I know who/what a user or a question is (resource), I know the media types(Representation) and the website itself provides you hyperlinks to navigate through the website. This pretty much represents HATEOAS and lends the architecture to decoupling of logic of client server, and this is what I mean by www i.e. the web is RESTful in itself. If STACKOVERFLOW was not built conforming to HATEOAS, and like most of the documented “REST API’s” these days, then instead of links, you would have to type in stackoverflow.com/questions/<id>, replace id with the Question.id and paste that on your browser. MEH, see what I mean?
  3. I am using JSON , that makes my API RESTful ==> Incorrect , if anything using JSON violates constraint 6 of REST, hence killing the fact that your API is RESTful
    • If you look at the HATEOAS component defined before, its clear that JSON is not Hypermedia, and hence one of the constraints is lost and essentially your API is not pure REST anymore. There is work going on this area – particularlyJSON Linked Data – http://json-ld.org/ that makes the attempt to confirm to HATEOAS.
    • So you could use XHTML to make your REST API Purely RESTful
  4. I should be using REST because SOAP sucks – or thats what they tell me.  ==> Incorrect
    • This might come as a shock to many who seem to have convinced themselves that REST is in someway the absolute holy grail of Webservices. Firstly REST and SOAP are two very different things. Here I go again – REST is a set of Constraints for good service architecture while SOAP is a protocol. Simple object Access Protocol. I think the whole REST is a protocol nonsense stems from the constant comparisons made between these two words. Its highly dependent on your business needs. SOAP is tightly coupling, both the client and server need to be clear on an interface of communication, but in large systems that have tightly coupled business logic and legacy applications, this is quite useful to have and certainly wouldn’t be needed to be replaced by REST.

So,

REST – Representational State Transfer – Set of Constraints for building scalable fault tolerant client server systems, architectural framework

RESTful – Webservices that use the REST architecture

SOAP – Simple object access protocol

But here’s the thing, its hard to follow all the principles of RESTful services to the letter, and in many cases harmful. But hopefully this clears up some of the myths of what defines pure REST API’s. Its not ok when developers basically follow the myths I’ve presented, and are in the dark about what RESTful services really are. As long as what developers build are HTTP API’s that follow some best practices(another post on this to follow just for best practices for writing API’s- RESTful or otherwise) things should be fine in the webservices world.

Bonus Reading and Credits:

A word on Richardson Maturity Model – http://martinfowler.com/articles/richardsonMaturityModel.html

And the original Fielding dissertation on  Rest – http://www.ics.uci.edu/~fielding/pubs/dissertation/rest_arch_style.htm#sec_5_2_1_1

SO links: http://stackoverflow.com/questions/19843480/s3-rest-api-and-post-method/19844272#19844272 and  http://stackoverflow.com/questions/19884295/soap-vs-rest-differences

REST Cook Book: http://restcookbook.com/

STARTUPS

Sam Altman’s CS183B – Lecture 6 – Alex Schultz

Lecture 6 was by Alex Schultz – a very good talk on Growth.

Salient points

  1. Self intro: Paid for college doing online marketing, directions marketing. I started with SEO in the 1990s. I created a paper airplane site, and had a monopoly in the small niche market of paper airplanes. When you want to start a startup also see how big the market could be. (In the long term, it wasn’t great.) But what that taught me was how to do SEO. And back in those days it was Alta Vista, and the way to do SEO was to have white text, on a white background, five pages below the fold, and you would rank top of Alta Vista if you just said planes 20 or 30 times in that text. And that was how you won SEO in the 1990s. It was a really, really easy skill to learn.
  2. Retention is the single most important thing for growth and retention comes from having a great idea and a great product to back up that idea, and great product market fit. The way we look at, whether a product has great retention or not, is whether or not the users who install it, actually stay on it long-term, when you normalize on a cohort basis, and I think that’s a really good methodology for looking at your product and say ‘okay the first 100, the first 1,000, the first 10,000 people I get on this, will they be retained in the long-run
  3. Every different company when it thinks about growth, needs a different North star; however, when you are operating for growth it is critical that you have that North star, and you define as a leader. That one metric that drives the company and a person who drives it.
  4. For Facebook the magic moment, is that moment when you see your friend’s face, and everything we do on growth, if you look at the Linkedin registration flow, you look at the Twitter registration flow, or you look at what WhatsApp does when you sign up, the number one thing all these services look to do, is show you the people you want to follow, connect to, send messages to, as quickly as possible, because in this vertical, this is what matters. When you think about Airbnb or eBay, it’s about finding that unique item, that PEZ dispenser or broken laser pointer, that you really really cared about and want to get ahold of. Like when you see that collectible that you are missing, that is the real magic moment on eBay. When you look on Airbnb and you find that first listing, that cool house you can stay in, and when you go through the door, that’s a magic moment
  5. Operating for growth, what you really need to think about, is what is the North star of your company: What is that one metric, where if everyone in your company is thinking about it and driving their product towards that metric and their actions towards moving that metric up, you know in the long-run your company will be successful.
  6. Make sure you have deliverability. Focus on notifications and triggered based emails, SMS, and Push Notifications.( A bunch of theory on SEO methods)

    Conclusion:

    “A good plan, violently executed today, is better than a perfect plan tomorrow.”

    BTW PG said Startup = Growth – so lets keep that in mind all the time shall we.

    Btw I highly recommend Genius and thank them for the lecture annotations. A lot of these posts are filtered from there- very helpful.

STARTUPS

Sam Altman’s CS183B – Lecture 5 – Peter Thiel

Been busy with office work and finally getting a chance to continue blogging on what I learnt with the Sam Altman lectures.

Lecture 5 was by Peter Thiel. Thiel is notoriously famous for being a cut throat execution oriented investor and has invested in some of the most successful startups on the planet. His lecture was of course polarizing (wouldn’t have it any other way) titled Competition is for losers and mostly took out pieces from his very successful and highly recommended book Zero to One: Notes on Startups, or How to Build the Future.

Salient points

1.)  If you’re the founder, entrepreneur, starting a company you always want to aim for monopoly and you want to always avoid competition. And so hence competition is for losers!

2.) If you have a valuable company two things are true. Number one, that it creates “X” dollars of value for the world. Number two, that you capture “Y” percent of “X.” And the critical thing that I think people always miss in this sort of analysis is that “X” and “Y” are completely independent variables, and so “X” can be very big and “Y” can be very small. “X” can be an intermediate size and if “Y” is reasonably big, you can still have a very big business.

3.) If we look at perfect competition, there’s some pros and cons to the world of perfect competition, on one end of the spectrum you have industries that are perfectly competitive and at the other end of the spectrum you have things that I would say are monopolies, and they’re much more stable longer term businesses, you have more capital, and if you get a creative monopoly for inventing something new, I think it’s symptomatic of having created something really valuable.

4.) Companies lie for their own benefit regarding monopolies. Anyone who has a monopoly will pretend that they are in incredible competition; and on the other end of the spectrum if you are incredibly competitive, and if you’re in some sort of business where you will never make any money, you’ll be tempted to tell a lie that goes in the other direction, where you will say that you’re doing something unique that is somehow less competitive than it looks because you’ll want to differentiate. (Example, a restaurant not making money in Palo Alto will position itself as “Well we’re the only British food restaurant in Palo Alto” to seek attention of the monopoly they create. CounterExample of Google who have monopolized the search market but all press and PR is about Self driving cars and the competitive space they are in)

5.) Very counterintuitive ideas that comes out of the monopoly thread is that you want to go after small markets. If you’re a startup, you want to get to monopoly. You’re starting a new company, you want to get to monopoly. Monopolies have a large share of the market, how do you get to a large share of the market? You start with a really small market and you take over the whole market and then over time you find ways to expand that market in concentric circles.

Conclusion:

So much of people’s identities got wrapped up in winning these competitions that they somehow lost sight of what was important, what was valuable. Competition does make you better at whatever it is that you’re competing at because when you’re competing you’re comparing yourself with the people around you. I’m figuring out how to beat the people next to me, how do I do somewhat better than whatever it is they’re doing and you will get better at that. I’m not questioning that, I’m not denying that, but there often comes this tremendous price that you stop asking some bigger questions about what’s truly important and truly valuable. Don’t always go through the tiny little door that everyone’s trying to rush through, maybe go around the corner and go through the vast gate that nobody is taking.

 Now if you haven’t been catching up, its fine, my final post in this series is going to be an infographic on what’s been talked about so far and I absolutely recommend his book Zero to one – Buy it here, must read for anyone wanting to build startups

Uncategorized

Cantonese Proverbs in One Picture

廣府話小研究Cantonese Resources

阿塗(Ah To), a graphic designer and part-time cartoonist who concerns about the survival of Cantonese in Canton and Hong Kong, has just published a comic called ” The Great Canton and Hong Kong Proverbs” on Hong Kong independent media “Passion Times“. The cartoon contains illustrations of 81 Cantonese proverbs.
comic58

“The Great Canton and Hong Kong Proverbs”
In 1559, Dutch artist Pieter Bruegel created the oil painting “Netherlandish Proverbs” which illustrates many Dutch proverbs to praise the Dutch culture. In 2014, Ah To imitated the idea and created “Great Canton and Hong Kong proverbs” illustrating 81 Cantonese proverbs to propagate the Cantonese culture and defend Cantonese.

81 Cantonese Proverbs 

answer0226

(The 2 extra proverbs are the bonus given by the author. And audio clips only work on PC.)

1. cantoneseproverb67 上山捉蟹 [séuhng sāan jūk háaih]

(To catch crabs on a hill)
Harder than hard, almost impossible

2. cantoneseproverb1 鬼揞眼 [gwái ám…

View original post 2,297 more words

STARTUPS

Sam Altman’s CS183B – Lecture 4 – Adora Cheung

Lecture 4 was on Building Product, Talking to Users, and Growing by Adora Cheung. Adora is the CEO of HomeJoy, a online platform which connects customers with house cleaners. Adora Cheung did the first few cleaning jobs herself(Talk about dogfooding), and until late 2013 continued to work at least one cleaning job per month. HomeJoy, After Series B, led by Google Ventures, in early December 2013,  has raised about $38 million totally so far.

Salient points from the talk.

  1. Build a startup when you have a big block of available free time!
  2. Build a startup that solves a problem you have. Adora and her brother Aaron started a company called PathJoy(online platform to connect users to life coaches), but didn’t continue after a years effort, since they realized it wasnt the problem they wanted to solve.
  3. Start by learning A LOT about the target segment, become experts,story board ideal user experience. Build an MVP and put it out, smallest feature set to solve the problem, with simple product positioning.
  4. Have lots of avenues for customer feedback. Go out, talk to users. But setup support@company , surveys, qualitative,quantitative feedback, beware of the honesty curve, graph everything. Stealth is stupid.
  5. There are three types of growth. Sticky, viral, and paid growth.

Sticky growth is trying to get your existing users to come back and pay you more or use you more. Viral growth is when people talk about you. So you use a product, you really like it and you tell ten other friends, and they like it. That’s viral growth. And the third is paid growth. If you happen to have money in the bank you’re going to be able to use part of that money to buy growth.

Key to growth = Sustainability

STARTUPS

Sam Altman’s CS183B – Lecture 3 – PG

Lecture 3 had  Paul Graham speaking on the Counterintuitive Parts of Startups, and How to Have Ideas. PG is an advisory icon and the founder and former president of YCombinator, and the primary driving force for this list of amazingly good startup’s. He is also known for his essays on startups.

PG spoke about how startup’s are counter intuitive.

1.)  Startups are counterintuitive! They are like skiing. The first counterintuitive point is that , You cant trust  your intuition about startups,but you can trust your intuition about people.

2.) The second counterintuitive point is that what you need to succeed in a startup is not expertise in startups, what you need is expertise in your own users. Most startups founders now go through the motion of building a startup, because it feels cool, hire people, rent an office in SoMa etc., But forget to do the one important thing, build something users want.

3.) The third counterintuitive thing to remember about startups: starting a startup is where gaming the system stops working. IN a large company, you can get away by sucking up to people, sending emails late nights and on weekends to create an illusion of hard work, but you cant do that in startups, because there is no boss/management to fool, and you are fooling yourself. To some extent with the right phrases you can fool some investors but its not in your interest to do so!

4.) The fourth counterintuitive point is that startups are all consuming but this doesn’t end once you grow as a company. They take over your life. As the companies get bigger, the problems don’t go away, they just change scale. Larry page, a billionaire, still has a lot of problems to tackle daily.  IT NEVER GETS EASIER. It maybe a different set of problems but still the total volume of worry never decreases. Mark Z cant bum around a foreign country

5.) The fifth counterintuitive point is that the age/point in your life at which you should start a startup will eventually come to you. Starting startups is hard, but the answer to the question when should I do it, you can tell!

Reading how to get ideas is here: An exercise left to the reader.

PG’s essay on the topic.

STARTUPS

How building startups are like planting trees

Starting startups and planting trees basically have the same steps. Here’s an analogy

1. Find the right soil and area –  this is basically market research without which there is no beginning.  Making sure the soil, the weather, the surroundings is crucial for plants to even begin. Same for startups. Find the right product/market/idea fit. Solve a real problem.

2. Buy the right seeds –  this is the formation of the idea. The idea has to be right, and fit the market, just like the right soil needs the right seeds. Also to buy the seeds you need money and possibly land. You either put in your own money or borrow from friends and family with a promise to repay in kind. This is angel investment. Angels can be friends and family/bootstrapped/angel investors.

3. Choose the right people to plant the tree with – planting and growing trees is a lifelong process.  And you need the right set of people to work with you, who share the same vision for what the tree should look like 20 years from now and will do anything to see it gets there. Also ideally you’ve planted other trees before with these people. This is how you should choose Co founders. The vision should be to grow an Orchard of trees. Ideally you’ll need someone who knows the science of planting trees (CTO) and someone who can ensure daily watering/healthcheck operations happen smoothly(COO). Your primary role(CEO) will be to oversee the other two and start talking to local markets about the benefits of buying your produce and signing up early customers. The purpose is after all to create customers.

4. Plant the seed – This is the start. Make sure everything is in order. Team is set, plans are set, laser focused on building, growth and execution.

5. Water the seed till it becomes a healthy plant – Early growth. This basically proves the land is fertile and the growing trees here is not disastrous. This also sends the right cues to investors and customers alike, who see that 20 years from now, you might be the best producer in the market, and want an early piece of it.

6. Find people who can buy you the right tools and fertilizers– expecting trees to grow on their own is hard especially considering all the factors that can go against it. Find Investors who can buy expensive fertilizers and help solve your problems with their vast experience of having grown trees before. These people are only here to help. Make sure of that, and that you don’t give up too much of your early produce or promise too much of your end product

7. Water everyday and grow the tree – Water the plants everyday, ensure good health, start selling produce, put money back into buying things that help in growth. Similarly for startups – execute, execute, execute, use early revenues to buy more tools , build more metrics and hire more people

8. Find people to help you plant more seeds – find coworkers who believe in your vision and work hard with you. Find people willing to invest in helping you to plant more trees. Scale your startup.

9. Build a business- now that you have the produce, the customers willing to buy your produce and the right hardworking team you have sustainable business. Grow the business. Start planting more trees and orchards, start planting trees in other suitable orchards, acquire orchards that have trees you’d like but cant plant them due to the time involved.

10. Sell the Orchard or continue to expand across regions, orchards – sell or go IPO

At every point be aware of bad weather conditions(co founder spats, bad hires), diseases(bad execution, bad advice) that can kill the plant and make sure to root them out early.

Very similar don’t you think. Comments?