Previously, I outlined my general understanding of FBS’s Spark Platform and the legal framework it creates. Now, as promised, I’m considering the technology framework. I should warn all readers, however, that I’m a lawyer, not a technologist, so I’m counting very heavily on tech-savvy folks to correct me in the comments where I go astray. It seems to me the big questions are (a) how data gets into Spark; (b) how it gets out for appropriate uses.
As I understand it, MLSs taking part in the Spark Platform will provide RETS feeds of listings data to FBS. (An exception may be the many participating MLSs who are already FBS customers; I expect there might be more efficient back-end approaches for FBS to copy and synchronize their listing data onto the Spark Platform.) The Spark Platform will, to the extent possible, map listings data from the MLSs’ native formats onto RESO’s data dictionary, making it as standard as it can be on a national level. The RESO data dictionary is a fairly new piece of standardization. (My colleague Mitch Skinner—@MitchellSkinner—has been working with RESO on the intellectual property framework and license agreements for the newly launched data dictionary this year.) The RESO dictionary is not meant, though, to cover every data field that might be important in a given MLS. I asked FBS CEO Michael Wurzer who will you handle local “oddities”—e.g., “dune grass” in western Washington State? He responded:
These will be available through the API as custom fields. As described here, data standardization through the platform is a process, which will improve over time as more applications are developed and need more standardized data and the dictionary broadens and improves.
But Spark seeks to go further than that; it would also extract and maintain contact information provided by agents. Thus, if FBS is your MLS vendor (FlexMLS system) and you put buyer prospects into the MLS system, I anticipate contact info for those prospects would be usable to you in apps you obtain through the Spark Platform. I also understand that FBS’s goal is that the searches you save in MLS would be available in these other apps, too. I wondered whether contact information and saved searches collected by an agent and put into one Spark-enabled app will be available to the agent in other Spark-enabled apps and even through the agent’s MLS. I asked FBS to confirm/elaborate on these matters, and here’s Michael’s response:
[T]his is the goal, but, importantly, standardization of searches is a process that’s going to take time similar to the standardization of fields. Some searches may be converted to the new standard fields and search format but others may not be able to be converted. Most importantly, however, new searches created by applications using the API, such as Flexmls Mobile, will be in a standard format and available to all applications through the API. Ending the days of user data being locked in properitary formats that cannot be accessed by other applications is a main goal of the Platform.
There is no standard comparable to RETS or the RESO data dictionary yet for these other types of data; though our friend Matt Cohen has long been urging the development of such standards for a variety of reasons.
Another kind of data comes from the MLS. The MLS indicates on the Spark Platform what data fields and statuses are confidential (i.e., cannot be displayed on the Internet), what data fields are required on IDX displays, what data fields are permitted in VOWs but not permitted in IDX, etc. This “metadata” is not data about listings and individuals, but data about how that data can be used. Incorporating these business rules into the Spark Platform is a critical step to create efficiency in development of products based on data from many different MLSs.
As for getting data out of Spark, that requires use of an API, or application programming interface. An API is basically a framework for two computer programs (or a database and a program, or two databases… you get the idea) to talk to each other. The API provides all the data and metadata I described above in one consistent fashion for all the MLSs involved in Spark to each product or service provider (PoSP)/developer in the Spark network. The data a PoSP gets will depend on what kind of application it is offering. (Less data for IDX than for VOW services, etc.) Documentation for the API is available now online; I confess that I don’t understand most of it, and I’m counting on more technical readers of this blog to comment on its sufficiency and practical utility.
Note that this technology framework also implements the legal framework I discussed in the last post. Spark is not the first to do this. RETS IQ has implemented an automated contracting framework for use with its custom RETS server projects in at least one of our client MLSs. (Mitch helped develop the contracts for that, too.)
I mentioned there may be some difference between Spark’s integration with FBS systems and its integration with non-FBS systems. Brian Boero also asked whether MLS vendors other than FBS would take part. As I understand it, FBS is working with several MLSs and their vendors to determine the best way to integrate into the MLS system the Spark Store and the applications purchased. The idea is that adoption will be improved by deeper integration into the MLS system, and so developing standards for this integration is critical as well. It seems to me that FBS is very open to working with the other system vendors on this; I don’t know how much they will want to cooperate.
Well, that’s a rudimentary understanding of the technology framework. Again, I hope that folks who understand the technology better will speak up with comments and clarifications. I know Mike Wurzer from FBS sometimes reads this blog, so if you direct technical questions his way in the comments, you can probably expect a pretty prompt answer.
Next time, we’ll consider what I think are the significant obstacles to the adoption of the Spark Platform (or similar concepts). See you then!
-Brian
It is my understanding that all of the data – MLS Data, Consumer Data, Agent Data, and Search Data would be in one data base accessible through the API.
Brokers and agents with access to more than one MLS would be able to have applications that span more than one. Is that true?
Brian, as you know and mentioned, I am a proponent of having a common way to access MLS data. The common “API” to access MLS data is called RETS. The RETS community has recently completed work on the data dictionary you described, is about to approve a new and easier way of accessing that dictionary (what is referred to as “transport”) and after that, contacts, saved searches, and prospects are (
Matt is correct. RESO currently has an active work group addressing the standardization of transport, and we are making great progress!
We have pulled together some of the brightest engineers from the industry to work on this project, and there is a great mix of MLS's and vendors represented (including FBS). The chair of this workgroup is Scott Petronis, from Onboard Informatics.
Victor: Maybe Michael will respond to your question.
Matt/Rebecca: Like you both, we are contributing our volunteer efforts to the RESO standards. We believe in them, too.
Matt's comment argues that FBS's efforts to deploy its own API undermine the efforts of RESO to deploy a standard one. I'm wondering if others will weigh in on that? I have a couple questions
At iMapp we use RETS to transport data and there are enough variations from MLS to MLS to make your head spin. Getting everyone standardized is an excellent goal.
The RETS standard is ideal as it has support from vendors, MLSs and NAR.
Getting the same level of support for a different, proprietary format may be extremely difficult, especially one that taxes the developer
Matt Cohen added the following comment on my Facebook posting of this blogpost:
Matthew wrote: "It's not enough for some of these proprietary APIs to "leverage" RETS – you either use RETS or you don't – the middle ground is a slippery space. Some vendors have said, "As RETS is expanded, we'll incorporate that into our API." That makes no sense – to
Victor Lund posted this additional comment over on RETechnology.com.
Victor wrote: I am a huge fan of Spark ideology. The thesis of an app store for real estate is excellent: secure data; real time on-off; synchronized branding and customer records. It is the way that all software should work.
I dream of an API that would be robust enough to power all of the data
Mike Sparr from Goomzee laid it out on my blog: "each MLS should not rush into adopting new standards from various vendors"
FBS is fully supportive of RETS and RESO and the contract we're executing with MLSs for the Spark Platform assures them that we'll make the Spark API compliant with RETS.
Importantly, however, the reality is that standards often are advanced by those who implement new approaches. Look at the standards for HTML. They were advanced by organizations such as Mozilla, Google and
Brian, I'll also endeavor to answer some of your questions:
1. First, might FBS's involvement with its own API and with RESO developments prove beneficial for the RESO efforts, because of FBS's real-world experiences with the approach?
FBS: Yes, we believe this is very true.
2. Second, might that dual role prove detrimental, especially if FBS were to
Victor: Brokers and agents with access to more than one MLS would be able to have applications that span more than one. Is that true?
FBS: Yes, the Spark API supports cross-MLS searches.
Michael, I respect the work you have done – and continue to do as a part of RESO. But I think this API is in conflict with a standards-based approach. Please don't characterize this argument as personal, about respect, because respect for you isn't something that will waver, even when we disagree about something like this.
Thanks for putting this series together Brian!
Since you asked, as a PoSP I firmly agree with Matt that (yet another) API would not solve my pain. In fact, it would increase my current costs as well as my future costs of keeping up with this API as new versions of the it are deployed.
While we have solved the problems of standardization internally, I welcome Matt (and
Alon, you're right that one of the primary benefits of the Spark Platform is for companies unlike yours that haven't already spent a ton of money to "solve the problems of standardization." Of course, that lack of standardization is one of the big barriers to entry we're trying to remove to increase competition and innovation in the space. And, like you, Matt, and Rebecca,
Setting aside the discussion about which API to use, I think it bears reminding that an API is merely a method of transferring (MLS) data.
By itself, an API is not more secure than any other method of accessing, downloading, (potentially) caching that data…and mis-using it in violation of MLS rules and licensing terms.
Now, my friend Rob Larson would say (and have
Thanks to Alon and Robert for chiming in. Are there other product and service providers out there willing to share their thoughts?
-Brian
Alon, you can read the terms of use starting at http://sparkplatform.com/docs/terms_of_use/developer. The Spark Platform integrates data licensing and MLS rule compliance into the API based on roles defined by the MLS. Further, the restriction on replication of the database reduces the amount of data exposed to misappropriation. You're right that bad actors will always exist and APIs can be
Brian, you ask, "Are there other product and service providers out there willing to share their thoughts?" I'm guessing most of the software developers who read MLS Tesseract are vendors like Alon and Robert who are already entrenched in the industry and so will be against the Spark Platform as it primarily promises increased competition. The promise of increased competition, of
One more comment. Though I disagree with the arguments raised in the comments so far, I'm the first to acknowledge there are significant obstacles to the success of the Spark Platform. For example, I worry a great deal about whether the data standardization process will respond quickly enough for developer needs. We've built some nice tools to help us and MLSs streamline the data mapping
Michael: I'm still hoping to hear from current developer participants in the market who ARE interested in pursuing the Spark platform, though I understand your explanation of why I might not.
All: Generally, many of the points made here so far relate not to whether the Spark platform technology will work, but to what the business consequences of adopting it are. I'm hoping to
By the way…. I must disclose that I am the principal investor in RE Technology – RE Technology launched an eCommerce solution called Success Store (TM) and a component solution called Success Tracker. It clouds my vision.
I didn't say I was against Spark, but I have concerns that I consider to be valid. Is it telling that none of those concerns were addressed in the 7 posts following?
While increased competition may not be welcomed by all, it ultimately makes for better products and a better market. Bring it on and let the users decide.
Will Spark allow 3rd party developers to bring an
Robert, your questions regarding the 30% transaction fee, advertising, and in-app purchases are all addressed in the developer agreement. In sum: Fee products are allowed; advertising and in-app purchases are not.
Regarding an alternate front-end to the MLS, that's really the whole point, so, yes, definitely allowed. Corelogic, Discover, and any other vendor are allowed to
Oops, meant to say "free" products are allowed.
Also, I vaguely recall answering all these questions from you on another blog. I think it was WAV Group. Anyway, maybe review those posts.
Thanks for the thoughtful (and patient) responses Michael,
I am not against the Spark Platform…on the contrary, if I can "outsource" my data aggregation and normalization costs and re-allocate my resources to product development, I would be able to better leverage my strongest competitive advantage – my deep domain expertise and access to the SF Bay Area innovation talent.<
This comment has been removed by the author.
I can understand the concerns of someone moving forward with their own version of an expected standard. Having different APIs built out there in differing ways certainly appears to be going in the wrong direction. however, if you look back in days of old, the development of a proof-of-concept was required before something could be adopted into the RETS standard. Also, as many have noted in this
Echoing Rob's sentiment, I think the Real Estate technology ecosystem benefits from FBS' efforts, regardless of whether it does so by spurring the standard to advance more rapidly or establishing a de facto transport standard in its own right.
Mike's advocacy for standardized data hasn't wavered in 8 years, and he continues to be committed to that process under the
Rob, proof of concepts are useful – but this isn't just something being done in a "lab" as an internal experiment, or cooperating with one or two third party developers that understand their efforts in the proof of concept may need to be scrapped. And at this point we have several MLS vendors moving forward with APIs – as I said in my original response, this is not just about what
Matt, I point you back to Rebecca's earlier comment that what you're asking for — collaboration at the RESO level — is already happening.
As that work is progressing, perhaps you should disclose your conflict of interest on this topic? http://clareitysecurity.com/2012/05/clareity-securitys-sso-portal-evolves-into-a-full-application-store/
Or, if that is not a
Michael, we're talking about whether the proliferation of APIs – again, not just yours – could cause a problem if not brought together under the standards effort. The fact that Clareity Security has an app store doesn't create a conflict in wanting to see the industry move forward on a unified standard. Of course I am glad you are collaborating with RESO, as are several other