How RDFa/Semantic Data can help bridge Drupal and Non Drupal content without losing appeal or context.
The problem
Search is a hard problem, it really is. Let me show this by using an example of a food chain that wants to add Drupal as their homepage of the whole chain. Of course, like many other organisations they do not only have Drupal running but also a subset of other web frameworks open and closed source systems.
Now, they wanted Drupal to become the front-page of all that content, but would you want to migrate all of this content in Drupal just so it becomes navigable so the Drupal site could redirect all content to the right site? Since we do not want our Drupal site, that will become our front page/portal of all this content, to directly reach out to all these other systems, because that would be impossible to scale and maintain, we are thinking of using Apache Solr as our Search Index to serve all this different content to our system.
A possible solution
One of the possible solutions is to convert all non-Drupal sites to Drupal sites and use the Apache Solr Multisitesearch module. By indexing all your content from all Drupal sites into the same Apache Solr index will allow you to search across all of them simply by using 1 query to the Apache Solr index. Since we know we are using Drupal and the module version is the same we know that we are capable of searching all this content in a similar way. We call this the Drupal way, since all knowledge on how the mapping between the Drupal fields and the Solr index is inside the Drupal module and is a "Drupalism".
What if we want to add content from non-Drupal sites?
That's a very good question, as a organisation that maintains multiple websites with different technologies you are not always able to switch all of them to the same platform or keep everything updated in a similar fashion.
Luckily the Apache Solr already did the groundwork to make this possible. The way the Apache Solr Search module works is that it is completely independent from the Drupal nodes on the website it is running. It will show you whatever is in the Apache Solr index as long as it follows some very basic structure :
- id
- site
- content
The Apache Solr Search module does not need much more than this information to just show data from other sources. But we now encounter another problem. We do want to see more contextual information than just a title and a snippet. We want to use all possible data and we can describe that as a mapping problem because the module translates field names to solr field names and this could wildly differ from site to site. Even if you only use Drupal sites!
Note: not all these fields are mapped in the same as they would be mapped for real, I invented a quick mapping but the concept should be clear. The solr fields that are used in the index are shown in Bold
An Example
Site 1
- Content type : Event
- Field : field_startdate -> dm_field_startdate
- field : field_enddate -> dm_field_enddate
- field : field_location -> ss_field_location
- field : field_description -> ts_field_description
Site 2
- Content type : Event
- Field : field_start -> dm_field_start
- field : field_end -> dm_field_end
- field : field_location_event -> ss_field_location_event
- field : field_body -> ts_field_body
Both sites have Drupal of a same version and have the same module version of Apache Solr Search installed, yet they are not able to share this information across systems as we lack a way of understanding what each field is. We are essentially missing the semantic data that describes this field in a generic way.
To make it even more complex I'll add third site with Events that is non-Drupal, say it's Wordpress and we prefix everything with WP because of some decision in the system. I do not claim that WP does any of this, it's for the sake of the argument ;-)
Site 3
- Content Structure : Event
- Field : wp_startdate -> dm_wp_startdate
- field : wp_enddate -> dm_wp_enddate
- field : wp_location_event -> ss_wp_location_event
- field : wp_descripton -> ts_wp_description
Even if we have a way to index this content in Solr via Wordpress, we do not have a shared agreement on how to name these Solr fields and so we miss out on tons of features such as sharing Facets to filter all of our content in a consistent way and we are not able to show consistent Rich Snippets.
Rich snippets are a way to show metadata in search results. Google has more info about them
Another possible solution using Semantic Data
Describe all of our structured content with Semantic Data such as schema.org and RDFa so that we can translate this in common field names and in effect translate this in common solr field names! By using the CURIE standard and the rdfx module (that requires the ARC2 library) that can translate CURIE uri's to full uri's we can now easily have a common way of figuring out how our fields should be named.
Site 1
- Content type : Event
- Field : field_startdate -> schema:startDate -> http://schema.org/startDate -> dm_httpschemaorgstartdate
- Field : field_enddate -> schema:endDate -> http://schema.org/endDate -> dm_httpschemaorgenddate
- Field : field_location -> schema:location -> http://schema.org/location -> ss_httpschemaorglocation
- Field : field_description -> schema:description -> http://schema.org/description -> ts_httpschemaorgeventdescription
Site 2
- Content type : Event
- Field : field_start-> schema:startDate -> http://schema.org/startDate -> dm_httpschemaorgstartdate
- Field : field_end -> schema:endDate -> http://schema.org/endDate -> dm_httpschemaorgenddate
- Field : field_location_event -> schema:location -> http://schema.org/location -> ss_httpschemaorglocation
- Field : field_body -> schema:description -> http://schema.org/description -> ts_httpschemaorgdescription
Site 3
- Content Structure : Event
- Field : wp_startdate-> schema:startDate -> http://schema.org/startDate -> dm_httpschemaorgstartdate
- Field : wp_enddate -> schema:endDate -> http://schema.org/endDate -> dm_httpschemaorgenddate
- Field : wp_location_event -> schema:location -> http://schema.org/location -> ss_httpschemaorglocation
- Field : wp_description -> schema:description -> http://schema.org/eventDescription -> ts_httpschemaorgdescription
Note: We include the schema.org url as we can have different sources for RDF metadata and these can mean different things.
By defining our structure in a standard way we now have uniformity in our Solr index field mappings and can do really cool things with this
What can you do already?
All of this is already possible if you install the rich_snippets module and use branch 1815744, I committed the test code to index and register facets based on these RDF mappings. An example site can be seen here : http://rdfsolr13test.devcloud.acquia-sites.com
Future possibilities
By having a crawler that implement this exact same mapping schema we can show external content from sites we don't maintain in our Drupal site. By imitating a content type in Drupal and define the mappings for those fields we are able to "identify" similar content and show those in the search index. This opens a lot of possibilities of bridging the gap between Drupal and non-Drupal sites without requiring a massive migration. It also places Drupal in a position that is suited for rapid development as content can quickly be brought available through the means of a shared Solr index.
There is still some work to make it easier, but I think we are already half way by defining what we want and by defining a standard on how we map rdf properties to solr field names and I've been thinking to make a very easy library for this so it can be common good and escape the Drupal island.
Thoughts? Ideas? Excited? Please leave a comment :-)
Comments
Dear Nick
Dear Nick
Thanks for this explanation, it's very clear!
Kind regards!
Very informative publish!!
Very informative publish!! Your suggestions are first-rate concerning motivation. Right here you've got posted anything new so i love your web publication. And also your article is unique for one and all readers. Really excellent to know about it. I'm tremendously happy so high-quality.
http://essayreviewratings.com/
I would like to claim that
I would like to claim that this writing is awesome!
It gives all necessary information on %BT%. I was seeking
something of that nature for a long time. It is clear that
you are competent copy writer so could I ask you for a little support?
Might you run through this particular overview Christa and
look at whether it is OK?
useful
useful
odnxlv <a href="https:/
odnxlv <a href="https://icbdoilonline.com/#">pure kana cbd oil</a>
Amoxicillin Feline Crf <a
Amoxicillin Feline Crf <a href=http://adrugo.com>cialis 5 mg</a> Amoxicillin And Asprine
Amoxicillin Feline Crf <a
Amoxicillin Feline Crf <a href=http://adrugo.com>cialis 5 mg</a> Amoxicillin And Asprine
Amoxicillin Feline Crf <a
Amoxicillin Feline Crf <a href=http://adrugo.com>cialis 5 mg</a> Amoxicillin And Asprine
Thanks for the sensible
Thanks for the sensible critique. Me and my neighbor were just preparing to do some
research about this. We got a grab a book from our
local library but I think I learned more clear from this post.
I'm very glad to see such wonderful information being shared freely out there.
Here is my web blog: Cat Hat - https://catswithbamboochopsticksforhands.com
Therefore, people spend more
Therefore, people spend more evening online.
my web site <a href="https://hi.mikronsindia.com/27/">लिंग की लंबाई</a>
Therefore, people select more
Therefore, people select more period online.
Feel free to surf to my web site ... تطويل الذكر - https://arqa.riz-sachsen.com/26/
Therefore, people direct more
Therefore, people direct more period online.
Here is my web page kā palielināt peni - https://lv.air-text.com/dzimumloceka-garums/637/ka-palielinat-peni/
Therefore, people direct more
Therefore, people direct more period online.
Also visit my weblog: kā palielināt peni - https://lv.air-text.com/dzimumloceka-garums/637/ka-palielinat-peni/
Therefore, people select more
Therefore, people select more second online.
my web-site :: heart tonic в аптеките -
https://bg.anotherplanetlighting.com/other/1618/hearttonic-v-aptekite-stranichni-efekti-bg-forum/
Therefore, people set aside
Therefore, people set aside more evening
Ways to make lots of money fast - How we earn money
online making money on internet - https://engb.aviron-monaco.com/ways-to-make-lots-of-money-fast/ .
I like what you guys tend to
I like what you guys tend to be up too. This type
of clever work and coverage! Keep up the great works guys I've included you guys to our blogroll.
Have a look at my web-site: diet and exercise plan to lose weight female, <a href="https://www.loseweightngainmuscle.com">loseweightngainmuscle.com</a>,
Thanks for your entire work
Thanks for your entire work on this web site. Betty take interest in going through investigations and it's obvious why.
I learn all about the powerful mode you render precious strategies on this web site and as well welcome contribution from visitors on this situation then our child is truly learning a lot.
Have fun with the rest of the new year. You're carrying out
a useful job.
my blog post :: <a href="https://lt.t2studios.net/potency/1854/vitaminai-potencijai-gerinti-kaip-sustiprinti-erekcija/">vitaminai vyrams potencijai</a>
Thanks for sharing this
Thanks for sharing this information. I really Like Very Much.
<a href="https://www.meritstep.com">online training institute</a>
Showing 1 to 25 of 500
Showing 1 to 25 of 500 Articles in Search Engines.
Here is my page - marketing advertising your website - https://web-promotion-specialists.pro/
Showing 1 to 25 of 500
Showing 1 to 25 of 500 Articles in Search Engines.
Have a look at my blog; marketing advertising your website - https://web-promotion-specialists.pro/
How To Get Priligy Real
How To Get Priligy Real Secure Ordering Fluoxetine Website Visa Accepted Pharmacy <a href=http://gemeds.com>kamagra venta con receta</a> Finasteride 1 Mg Prices Filling A Prescription For Viagra Free Shipping Acticin Where To Buy Overseas Next Day
Touche. Sound arguments. Keep
Touche. Sound arguments. Keep up the great spirit.
my blog post Time - http://www.timehall.com/time/United_Arab_Emirates/P33
Touche. Sound arguments. Keep
Touche. Sound arguments. Keep up the great spirit.
my web blog: Time - http://www.timehall.com/time/United_Arab_Emirates/P33
Good post. I check out your
Good post. I check out your blog pretty often, and you are
constantly coming up with some decent staff. I shared this blog post on my Twitter, and my followers loved it!
Cheers.
Review my site <a href="https://philemon.blog.wox.cc/">Denisov L</a>
Hello! I understand this is
Hello! I understand this is sort of off-topic but I needed to ask.
Does managing a well-established blog like yours take a massive amount work?
I am completely new to operating a blog but I do write in my journal on a daily basis.
I'd like to start a blog so I can share my experience and feelings online.
Please let me know if you have any suggestions or tips for brand new aspiring bloggers.
Appreciate it!
Here is my webpage :: <a href="https://blog.unitedheroes.ch/en/node/179?page=197">Moses Tikhonov</a>
Direct Amoxicilina Purchasing
Direct Amoxicilina Purchasing <a href=http://cialgeneri.com>cialis 20mg price at walmart</a> Achat Cialis Super Active 24h Effets Secondaire Cialis
Direct Amoxicilina Purchasing
Direct Amoxicilina Purchasing <a href=http://cialgeneri.com>cialis 20mg price at walmart</a> Achat Cialis Super Active 24h Effets Secondaire Cialis
Direct Amoxicilina Purchasing
Direct Amoxicilina Purchasing <a href=http://cialgeneri.com>cialis 20mg price at walmart</a> Achat Cialis Super Active 24h Effets Secondaire Cialis
Comprar Cialis Por Telefono
Comprar Cialis Por Telefono <a href=http://ausgsm.com></a> On Line Pharmacies
Comprar Cialis Por Telefono
Comprar Cialis Por Telefono <a href=http://ausgsm.com></a> On Line Pharmacies
Comprar Cialis Por Telefono
Comprar Cialis Por Telefono <a href=http://ausgsm.com></a> On Line Pharmacies
Comprar Cialis Por Telefono
Comprar Cialis Por Telefono <a href=http://ausgsm.com></a> On Line Pharmacies
Great writeup. I checked out
Great writeup. I checked out your blog site quite regularly,
and you're continuously coming up with some good staff. I shared this post on my Tumblr, and my followers loved it!
Good luck for the future.
my web site; Tigran at Web - https://juliusmordvinov.wixsite.com/blog
Great writeup. I checked out
Great writeup. I checked out your blog site quite regularly,
and you're continuously coming up with some good
staff. I shared this post on my Tumblr, and my followers loved
it! Good luck for the future.
Review my homepage: Tigran at Web - https://juliusmordvinov.wixsite.com/blog
Cialis Preise In Deutschland
Cialis Preise In Deutschland Viagra in polen kaufen Online Propecia Generic <a href=http://drugsir.com>cialis no prescription</a> Prix Du Cytotec Dans Une Pharmacie Dosage Amoxicillin Comprar Alli Adelgazante Online
Cialis Preise In Deutschland
Cialis Preise In Deutschland Viagra in polen kaufen Online Propecia Generic <a href=http://drugsir.com>cialis no prescription</a> Prix Du Cytotec Dans Une Pharmacie Dosage Amoxicillin Comprar Alli Adelgazante Online
Cialis Preise In Deutschland
Cialis Preise In Deutschland Viagra in polen kaufen Online Propecia Generic <a href=http://drugsir.com>cialis no prescription</a> Prix Du Cytotec Dans Une Pharmacie Dosage Amoxicillin Comprar Alli Adelgazante Online
Cialis Preise In Deutschland
Cialis Preise In Deutschland Viagra in polen kaufen Online Propecia Generic <a href=http://drugsir.com>cialis no prescription</a> Prix Du Cytotec Dans Une Pharmacie Dosage Amoxicillin Comprar Alli Adelgazante Online
Theгe is certainly lots to
Theгe is certainly lots to find out about tһiѕ topic.
I like all the tips you gave.
My web-site - <a href="https://antoniobrowns.webflow.io/">Antonio Brown</a>
Thеrе'ѕ defіnitelʏ a ցreat
Thеrе'ѕ defіnitelʏ a ցreat deal to find οut on this matter.
I juѕt like all the pointѕ үou havｅ made.
Visit mｙ homepage <a href="https://dorofeimishin.sitey.me/">Dorofei Mishin</a>
Howdy! This blog post couldn
Howdy! This blog post couldn't be written much better! Going through this article reminds me of my
previous roommate! He always kept talking about this.
I will forward this information to him. Pretty sure he will have
a very good read. Thanks for sharing!
My web site ... Federer news, <a href="https://steam-akk.ru:443/entry.php?b=65">https://steam-akk.ru:443/entry.php?b=65</a>,
Your approaｃh is very special
Your approaｃh is very special in contгast to other folks I have checked stuff from.
Thanks for sharing when you have gоt the time, suppose I am just going to takｅ notes from thіs iԁea.
Have a look at my web site <a href="https://coffeetables.jimdosite.com/">Rodion Yevseyev</a>
من آسوده خاطر هستم که داخل آزگار اینترنت مطلب قرین
بوسیله این پیام وجود ندارد چون مطلب تمام و کمال یافته ی پیرامون
این مبحث است
Also visit my website: <a href="http://www.majalesafar.com/websites/drtorabian/">هزینه اصلاح طرح لبخند</a>
عالی است مطلب تالیف شده توسط شما وافر لثه گردآوری شده است و من مطمئن هستم که این پست
از باب مفرط از وبلاگ نویسان دلپذیر است
Here is my site; <a href="http://diyarmirza.ir/1398/07/%d8%af%d9%86%d8%af%d8%a7%d9%86%d9%be%d8%b2%d8%b4%da%a9-%d8%b2%db%8c%d8%a8%d8%a7%db%8c%db%8c-%d8%b4%d8%a7%d9%85%d9%84-%da%86%d9%87-%d8%a7%d9%82%d8%af%d8%a7%d9%85%d8%a7%d8%aa%db%8c-%d9%85%db%8c%e2%80%8c/">دندانپزشک خوب در تهران</a>
عالی است مطلب تالیف شده توسط شما وافر لثه گردآوری شده است و من مطمئن هستم که این پست از باب مفرط از وبلاگ
نویسان دلپذیر است
Visit my site ... <a href="http://diyarmirza.ir/1398/07/%d8%af%d9%86%d8%af%d8%a7%d9%86%d9%be%d8%b2%d8%b4%da%a9-%d8%b2%db%8c%d8%a8%d8%a7%db%8c%db%8c-%d8%b4%d8%a7%d9%85%d9%84-%da%86%d9%87-%d8%a7%d9%82%d8%af%d8%a7%d9%85%d8%a7%d8%aa%db%8c-%d9%85%db%8c%e2%80%8c/">دندانپزشک خوب در تهران</a>
معنای مطلب ی این نبا از نظر خیلی
از افراد کم قیمت و کامل نیست ولی برخی از محتواها سکوی پرتاب موشک ها و بلاگ ها پیرامون
این مطلب در وب وجود دارند که می توان از آنها استفاده بهتری داشت
Here is my blog post; <a href="https://www.farsnews.com/news/13980713000992/%D9%88%DB%8C%DA%98%DA%AF%DB%8C%E2%80%8C%D9%87%D8%A7%DB%8C-%DB%8C%DA%A9-%D8%AF%DA%A9%D8%AA%D8%B1-%D8%AF%D9%86%D8%AF%D8%A7%D9%86%D9%BE%D8%B2%D8%B4%DA%A9-%D8%AE%D9%88%D8%A8">بهترین دندانپزشک زیبایی</a>
معنای مطلب ی این نبا از نظر خیلی از افراد
کم قیمت و کامل نیست ولی برخی از
محتواها سکوی پرتاب موشک ها
و بلاگ ها پیرامون این مطلب در وب وجود دارند
که می توان از آنها استفاده بهتری داشت
Stop by my site :: <a href="https://www.farsnews.com/news/13980713000992/%D9%88%DB%8C%DA%98%DA%AF%DB%8C%E2%80%8C%D9%87%D8%A7%DB%8C-%DB%8C%DA%A9-%D8%AF%DA%A9%D8%AA%D8%B1-%D8%AF%D9%86%D8%AF%D8%A7%D9%86%D9%BE%D8%B2%D8%B4%DA%A9-%D8%AE%D9%88%D8%A8">بهترین دندانپزشک زیبایی</a>
کهتر نمی توانم با این مقاله مشاجره یا موافقت کنم زیرا علم کافی تو
زمینه این مقاله و بلاگ ندارم
Here is my webpage :: <a href="https://www.ilna.ir/%D8%A8%D8%AE%D8%B4-%D8%A8%D9%86%DA%AF%D8%A7%D9%87-%D9%87%D8%A7-206/790022-%D8%A2%D8%B4%D9%86%D8%A7%DB%8C%DB%8C-%D8%A8%D8%A7-%D8%B4%D8%A7%D8%AE%D8%B5-%D9%87%D8%A7%DB%8C-%D8%A8%D9%87%D8%AA%D8%B1%DB%8C%D9%86-%D8%AF%D9%86%D8%AF%D8%A7%D9%86%D9%BE%D8%B2%D8%B4%DA%A9-%D8%B2%DB%8C%D8%A8%D8%A7%DB%8C%DB%8C">هزینه لمینت دندانی</a>
کهتر نمی توانم با این مقاله مشاجره
یا موافقت کنم زیرا علم کافی تو زمینه این مقاله و بلاگ ندارم
Review my homepage ... <a href="https://www.ilna.ir/%D8%A8%D8%AE%D8%B4-%D8%A8%D9%86%DA%AF%D8%A7%D9%87-%D9%87%D8%A7-206/790022-%D8%A2%D8%B4%D9%86%D8%A7%DB%8C%DB%8C-%D8%A8%D8%A7-%D8%B4%D8%A7%D8%AE%D8%B5-%D9%87%D8%A7%DB%8C-%D8%A8%D9%87%D8%AA%D8%B1%DB%8C%D9%86-%D8%AF%D9%86%D8%AF%D8%A7%D9%86%D9%BE%D8%B2%D8%B4%DA%A9-%D8%B2%DB%8C%D8%A8%D8%A7%DB%8C%DB%8C">هزینه لمینت دندانی</a>
Lowest Price For
Lowest Price For Amitriptyline Levitra Im Preisvergleich Farmaco Kamagra <a href=http://vhsfp.com>online pharmacy</a> Cialis 20 Mg Madrid Propecia Posologia Sin Canadadrug
Pages
Add new comment