Sometime today, at Mix, Microsoft is announcing project “Astoria.” While it may not be exactly pitched as this (I’ll have to see what the final announcement has), to me, this is Microsoft’s entry into the REST server and (more importantly) service area. As the new site says:
The goal of Microsoft Codename Astoria is to enable applications to expose data as a data service that can be consumed by web clients within a corporate network and across the internet. The data service is reachable over HTTP, and URIs are used to identify the various pieces of information available through the service. Interactions with the data service happens in terms of HTTP verbs such as GET, POST, PUT and DELETE, and the data exchanged in those interactions is represented in simple formats such as XML and JSON.
This is still all at the “project” status, so like one of the projects I find fascinating, Rational “Jazz,” don’t expect a delivered product anytime soon.
That said, the conversation and “playing around” with Astoria is possible. Long-time readers know that I’m a fan of releasing an alpha/beta/whatever early over big bangs. There’s a CTP available (early access to the code), a website, and running instance to play around with at http://astoria.mslivelabs.com/OnlineService.aspx.
Also, check out this post from one of the Astoria leads, the impressive Pablo Castro.
Background on the SDR
While I’m not at Mix (I had to decline the invitation for other travel: tragic!), earlier this month (10 April 2007), I had the pleasure with several other folks to sit in on a day-long pre-briefing (an SDR) on the topic with several other bloggers, consultants, and coders. I actually enjoyed this session quite a lot: it was much interactive, lively, and fun. Meeting the other non-Microsoft in attendance was great, and meeting the Microsoft tech leads, architects, community manager, and other folks around Astoria was great too.
As with the Blogger’s Corner at SAP, this kind of in-depth, get down and dirty stuff is just the right complement for the higher-level, strategy talk that I do in the more “big A” Analyst meetings. Don’t get me wrong, one side isn’t “better” than the other: but having both vantage points is key to a solid understanding of the given topic.
Enough context, on to the fun stuff.
REST Server?
When I say “REST server and service,” I mean software that provides for the storage, access, and searching over “resources,” or “entities” as Microsoft calls them. Putting it another way, it’s CRUD and the equivalent of “O/R Mapping” (like Hibernate) for a REST-mindseted design. The service implies that (a.) you access these operations over a network, and, more importantly, (b.) there’s a hosted, SaaS option (at http://astoria.mslivelabs.com). Now, point (b.) is just an experiment in the case of Astoria: they’re trying it out and seeing how it works. Think something like Amazon S3, but with semantics layered on-top.
Vocab
Now, the funny thing about Astoria, as with most big vendor approaches to a technologies unfolding in The Wilds of the The Web, is that Astoria uses it’s own vocabulary for what we’d call REST or “public web SOA” terms. This is actually fine, as that field is in flux at the moment anyhow. Besides, locking down names and terminology is a slippery path to WS-Deathstar 2.0, this time starting with REST as a base.
Entities
The sort of core data-model for Astoria is the ADO.NET Entity framework. An “entity” in this world is, again, similar to the “object instance” you’d deal with in code once an O/R mapper had sucked out that data from your database and constructed the object model for it. The difference here is that entities are more “prime things” unto themselves rather than just instances of objects. That is, the hope is that they’re more than just serialized object graphs.
As you can see, an “entity” here is essentially what we’d call a “resource” in REST-land.
URLs
The other, exciting thing about Astoria is that it’s a URL-driven design. That is, entities and the methods of interacting with those entites can be represented as URLs. Each entity can be uniquely identified with a URL (sometimes more than one URL it seemed) and you can do the old “URL as command line” trick where you search via URLs.
Now, the Astoria team has been noodling on the exact syntax to do this identifying and searching. The examples we saw were quite similar to XPath, but by no means exactly the same. As you can imagine, there’s all sorts of horky things to stress out about when figuring out the syntax to use in URLs: there are encoding issues, characters already used, and many different understandings of what “feels right” when it comes to URLs.
As an example, here are some Astoria URLs:
http://localhost:50604/northwind.ashx/Customers/[City eq 'London']/Orders[Freight lt 1]– Searching for customers in London with less than one freight order.http://localhost:50604/northwind.ashx/Customers[ALFI]– searching for the customer with the ID ofALFIhttp://localhost:50604/northwind.ashx/Customers[ALFI]?$expands=Orders– searching for the customer with the IDALFI, and pulling back all the Orders for that customerhttp://myserver/data.svc/Customers?$skip=30&$take=10– paging through results, from this paper
As the URLs imply, these entities are from the venerable Northwind database.
The URLs are used as identifiers for entites, but also for queries over the entities. As such, there are several “operations” you can pass through the URL:
$orderby– as in SQL$skipand$take– just pull in some # of entities, as for paging through result sets$keyset– just return the URL Id’s for the entities instead of all the data, for performance$expand– “drill down” the data in the entity graph$format– return either XML, JSON, or XML RDF$callback– use a JSONP callback
Now, we had quite a lengthy discussion about all of this syntax. Like I said, there’s still discussion and decision to be had around it. Part of getting the conversation started up now is getting help honing this URL syntax. While there’s some of it that I don’t like, I always thought XPath seemed weird until I got to know it, and then I liked it. Granted, XPath statements are not URLs, but they’re sort of cousins to my loosey-goosey coders mind-set.
We had a lengthy discusion about using the POST contents of an HTTP request to encode queries and other info. My take was: while this sort of breaks the ideas of REST and “disses” a purer URL approach, sometimes it will be a more pragmatic, if even elegant approach to a gorked up URL that makes less sense than old Amazon URLs. But, using POST could well be a slippery slope into subverting the potential of the URL as command-line and identifier.
The important point here is that so much thought is being put into using URLs in the first place. And, yes, I realize I’m probably slinging “URL” around when I should be using “URI” sometimes. <shrug>
SaaS
The point all of us at the SDR hammered home was the need to host this as a service on the public web. While there’s tremendous value in having Astoria behind the firewall, it’s sort of the very nature of this technology that it be hosted online as well. The whole point of Astoria for me is to be a thin, REST-y data-layer on-top of the cloud. The web is The Cloud, of course.
Not only is is “the whole point” of Astoria to be on the public web, but that angle would actually alleviate a significant part of the biggest problem for Astoria: it’s from Microsoft. The early adopters and promoters who could help sling-shot Asotira into wider success are not the biggest fans of Microsoft. They’re especially not the kind of people who want to run Microsoft middleware and databases (such as Astoria itself and the SQL Server instance required).
Hosting it on the web, where what’s running in the back-end (theoretically) doesn’t matter is a nice way around the deployment problems. Long-term, I’d also hope it would help keep the interoperable design as honest as possible: if traditionally non-Microsoft developers are using Astoria, the Astoria team will find the incompatibilities between the Microsoft and non-Microsoft worlds faster.
Data Security and Privacy
The other elephant in the room, long-term, is one of data security. This is the same issue that keeps some people from using Google. That is, the fear that the provider of a SaaS data-service will snoop around in your data and use it against you or, at least, to annoy you with ads and other “targeted” money-extraction schemes. Here, the road is difficult and tedious all around for everyone.
Ultimately, I’d like to see an approach where the data is somehow encrypted such that the provider couldn’t ever see the data. Really, that’s sort of impossible to garantee as the service needs to see the plain-text at some point. No matter what, you have to trust your SaaS provider and human-to-human trust isn’t something that can be coded away.
So What?
I could go on about the technical parts of Astoria. We had a great, day-long dive into it. But, Astoria is out there for you to go poke at and play around with.
If you’re interested in REST, SOA (hopefully, of the REST type), SaaS, or (though I shudder to type it) “the semantic web,” take a look at Astoria yourself. Like I said, it’s all at the “labs”/”project” stage now. But, that means there’s a chance to get in there and influence what the final release turns out to be. Does the interface and use model “work”? Do you like it?
If it weren’t for the hosting angle, I wouldn’t recommend people outside of the Microsoft ecosystem looking into it beyond the theoretical angle. But, since the Astoria team has been thinking and is experimenting with hosting it, I can see that it’d have wide applicability, esp. people making web applications, web/desktop hybrids, and other network enabled, SaaS software.
As my heavy SaaS angle implies, indeed, my prime advice for Microsoft here is to go at the hosted angle whole hog: just bite into that and hold on. That’s the lightening rail to grab onto for this kind of thing and what I feel the industry as a whole needs to move towards more aggressively.
Here, the primary issue beyond the interface and use model is scaling and performance. Going forward in a SaaSy world, services that can scale and perform are the highest value items that customers will pay for. “Anyone” can and will startup an HTTP server, do some crude resource/relational database mapping, and start providing data. What few people have the skill and capital to do, though, is make sure that stuff runs at an acceptable speed with near 100% uptime. The Astoria model seems far from “crude,” of course: the point is that dependability and “just works” is of incredible value, and ignoring those angles would detract from any other positive aspects of a project like Astoria.
Dare I invoke the “e-word”? Enterprise.
Disclaimer: parts of Microsoft are clients. Microsoft paid for my trip to the SDR, putting my a swanky hotel (it had a fire-place!). IBM is a client as well.
Technorati Tags: astoria, microsoft, saas, rest, services, ado.net, mix07, soa
Nice one Cote, really interesting article, It would be great to see Microsoft coming up with something that competes and even goes beyond EC2/S3 for modern developers.
Actually I though Microsoft would announce something like this at Mix07, but they are much further behind than I originally anticipated, my bad.
Until then I will stick with the EC2/S3 + web scripting tools for our builds.
P.S. It should be S3 not S2 in your post.
regards
Al
Glad you liked it. Thanks for the S3-S2 correction, I'll fix it. Clearly, got my EC2 and S3 blended up… 😉
Cote, interesting take from microsoft. Unfortunately, I don't see how it's better than xml-rpc. Yes, I did look at the word documents that show the api samples… It's still too enterprisey for me though.
It does look better than SOAP and WS-Friends, but not by much.
Hmm… it's like, totally out of left field to me. I mean, I *think* I like that enterprise guys are getting on board. I just think REST needs to grow as a wild and wooly thing for a while before it becomes tamed though. Premature standardization has hurt many a technology, y'know.
The thing that worries me most about this though, is actually the most interesting thing that it looks like MS is doing with their REST: Those XPath like queries. I think that if the syntax is actually sort of close to relational algebra, then yeah, that's probably pretty good. If they start going crazy and adding any sort of conditional that they want, then it's gonna be a nightmare and bad for REST in general.
Christopher Mahan: ah, the spector of XML-RPC. Sort of running around in the "nobel savages" part of the house, huh? I remember everyone totally dug XML-RPC several years ago and then jumped to the loosey word of XML over HTTP/REST. I don't think XML-RPC is a perfect substitute for REST-think. But, I do know several ISVs that use XML-RPC to great effect, sort of skirting the whole SOAP/REST arm-wrastle.
Danno: your comments on the URI/URL give voice to my own thoughts. Those operators could start to get out of hand. But, what if they didn't? Would that work? Alex Barnett has an more examples laden post for more reference.
@cote: I agree that REST and XML-RPC are different, REST being better for ease and variety of client solution (you can call on a REST-full service with wget and curl easily), but Astoria adds enough complexity that in my opinion, XML-RPC would work better: Variable in programming language -> xml -> variable in another programming language: perfect for program to program over web, with just 1 url (that of the service).
As far as retrieving large data sets, ADO-like, then XML-RPC is less efficient. But the question is: why are you even doing that? Why do you get 30,000 records? What operation will you do on these? What is is that the client will do that the service/server can’t? Do you need a summary? Do you need a total? Do you need a count? Do you need row 300 to 350 to display in a report? I think SQL already covers that. If you just need one number, ask the server for that one number.
I just think moving large chunks of data around is a mistake, and that will lead to bloat and potentially worse data security breaches.
That’s why I say XML-RPC is better as far as I am concerned. More UNIXy.
As far as XML over HTTP, yeah, I agree, but that was generally for the browser, and that was because the xml-rpc javascript libs were not all that great (the ones that worked 100%) were slow. Using a standard xml parser was faster (or so it seemed). However, roll-your-own xml tree walking was/is hard, so JSON came about, giving easy access to the variables returned to the javascript interpreter (bypassing the whole xml parsing).
The problem with JSON (not a problem if you’re careful–but we’re talking web developers here) is that you’re eval-ing an expression, and if that expression comes from another server upstream, you’ve got to be absolutely sure you can trust it. With XML-RPC, the browser js libraries are now robust and FastEnough that you can do js var -> xml -> server language (perl, ruby, python, asp, .Net, php, java) -> xml -> js var with just a few lines of code, and you don’t have to walk the xml tree, and you don’t have to trust an eval.
Am I offtopic yet? I’ll stop.
Christopher Mahan: not off topic at all. I really appreciate all your comments 😉
Nice post.
…Cote'(Redmonk) walks through Astoria, and recommends making a Microsoft go whole hog after turning Astoria into an SaaS platform…
Based on what I have learned since yesterday, so take it for what it is worth, I see two problems with the Astoria toolkit so far. The first is the use of spaces in the urls. Surely this is a major no-no. Sure we can replace the spaces with %20 but isn't one of the intents to make the url somewhat legible.
There must be valid alternatives to the space!
The second part that I really don't like is the inclusion of the client side proxy class generators. I think the toolkit should stop at delivering XML/json to the client. By generating classes on the client the toolkit is guiding developers back to the problems of shared types and remote objects. We don't want to go back there.
Darrel Miller: agreed on the spaces, and I see what you're saying about client side class generation. Did you find a place to input your concerns? The team seemed open when I talked with them.
The System.ServiceModel.Web that is included in the new Biztalk Services SDK is much closer to what I want to deal with. It seems to be a thin layer over the basic WFC plumbing that makes creating REST apps extremely simple.
Thank You, Only kim kardashian sex tape [url =http://www.kimkardashiansextape.350.com/#1]Only kim kardashian sex tape[/url], itn,
Your Site Is Great!, http://blog.bitcomet.com/post/304314/ Discount kim kardashian sex tape, oego,
Nise site, ray j sex tape information [url =http://blog.bitcomet.com/post/304397/#1]ray j sex tape information[/url], =(,
Thank You, Discount r kelly sex tape [url =http://www.rkellysextape.350.com/#1]Discount r kelly sex tape[/url], 87404,
:-(, Best celebrity sex tapes [url =http://www.celebritysextapeser.350.com/#1]Best celebrity sex tapes[/url], 8],
I bookmarked this guestbook., britney spears sex tape discount [url =http://www.britneyspearssextape.350.com/#1]britney spears sex tape discount[/url], tbx,
I bookmarked this guestbook., http://www.carmenelectrasextape.350.com/ carmen electra sex tape online, 427537,
I have the same., http://blog.bitcomet.com/post/304501/ Best carmen electra sex tape, 8]],
Thank you, All about jessica simpson sex tape [url =http://www.jessicasimpsonsextape.350.com#1]All about jessica simpson sex tape[/url], 00428,
Nise site, lindsay lohan sex tape information [url =http://www.lindsaylohansextape.350.com#1]lindsay lohan sex tape information[/url], 384584,
Hi, http://blog.bitcomet.com/post/304572/ Only lindsay lohan sex tape, >:]]],
Perfect work, http://blog.bitcomet.com/post/304584/ keeley hazell sex tape, :DD,
Great, http://blog.bitcomet.com/post/304596/ noelia sex tape discount, 8-DDD,
Nise site, http://www.parishiltonsextape.350.com/ paris hilton sex tape free, 8-O,
:-(, http://blog.bitcomet.com/post/304354/ paris hilton sex tape, :-[,
I have the same., http://blog.bitcomet.com/post/304607/ vanessa hudgens sex tape discount, 057508,
best for you, http://www.adrianalimasextape.350.com/ adriana lima sex tape discount, :(((,
Perfect work, http://blog.bitcomet.com/post/304671/ adriana lima sex tape free, 3483,
So where it to find, Buy amy fisher sex tape [url =http://blog.bitcomet.com/post/304680/#1]Buy amy fisher sex tape[/url], upkjac,
Great, web cam sex [url =http://blog.bitcomet.com/post/304687/#1]web cam sex[/url], =-DDD,
best for you, granny sex [url =http://www.grannysex.350.com/#1]granny sex[/url], 8-)),
Beautiful site, http://www.shemalesex.350.com/ Cheapest shemale sex, ksowvs,
:-), jessica simpson sex tape here [url=http://blog.bitcomet.com/post/304553/#1]jessica simpson sex tape here[/url], %-PP,
Excellent site. It was pleasant to me., shemale sex for you [url =http://blog.bitcomet.com/post/304700/#1]shemale sex for you[/url], 773380,
Only, http://www.bbwsex.350.com/ bbw sex price, 8-),
I bookmarked this guestbook., bbw sex [url =http://blog.bitcomet.com/post/304707/#1]bbw sex[/url], xsrt,
:-), http://www.badcreditstude.350.com/ bad credit student loans free, 0203,
Your Site Is Great!, bad credit student loans now [url =http://blog.bitcomet.com/post/304712/#1]bad credit student loans now[/url], mvzssf,
I like your work!, Best no credit check student loan [url =http://blog.bitcomet.com/post/304716/#1]Best no credit check student loan[/url], :-],
:-(, All about high risk personal loans [url =http://www.highriskpersonal.350.com/#1]All about high risk personal loans[/url], jkhk,
Your Site Is Great, All about high risk personal loans [url =http://blog.bitcomet.com/post/304720/#1]All about high risk personal loans[/url], %-[,
Hi, no credit check loans [url =http://www.nocreditcheckl.350.com/#1]no credit check loans[/url], iumvgh,
Real, http://blog.bitcomet.com/post/304723/ Best no credit check loans, zlv,
:-), Cheap student debt consolidation loans [url =http://www.studentdebtconsolidation.350.com/#1]Cheap student debt consolidation loans[/url], vfpkc,
Very interesting sites., Discount student debt consolidation loans [url =http://blog.bitcomet.com/post/304729/#1]Discount student debt consolidation loans[/url], dqcy,
I like it so much, http://www.need5000loanovernight.350.com/ need 5000 loan overnight, 318527,
Nise site, http://blog.bitcomet.com/post/309274/ need 5000 loan overnight, =DD,
So where it to find, personal loan for poor credit [url =http://www.personalloanforpoorcredit.350.com/#1]personal loan for poor credit[/url], doypln,
Help me to find the, http://www.mortgageratesr.350.com/ Cheap mortgage rates predictions, gloinj,
Best Wishes, mortgage rates predictions [url =http://blog.bitcomet.com/post/309300/#1]mortgage rates predictions[/url], ckc,
Is it so important?, http://www.bactrim.350.com/ Best bactrim, :-(((,
Excellent site. It was pleasant to me., http://blog.bitcomet.com/post/309308/ bactrim discount, =-]],
It is a very good thing, http://www.darvocet.350.com/ darvocet now, 543769,
Perfect work, http://www.percocethere.350.com/ percocet information, 8[[,
Nice, http://blog.bitcomet.com/post/309328/ percocet, 619,
Good Job, http://www.diovanhere.350.com/ diovan here, 513,
Help me to find the, diovan discount [url =http://blog.bitcomet.com/post/309341/#1]diovan discount[/url], oycetk,
best for you, http://blog.bitcomet.com/post/309349/ buy viagra discount, 8-PPP,
:-), Buy buy xanax [url =http://www.buyxanaxes.350.com/#1]Buy buy xanax[/url], %OOO,
Very interesting sites., http://blog.bitcomet.com/post/309413/ First buy xanax, wdgcz,
What’s the interest rate on this account? preteen pics
mjoeeo
I don’t know what I want to do after university pthc boys
%]]