json


What is difference between Post Tool and Index Handlers?


I have a JSON documents which needs to be indexed in Solr. The document looks like this:
{
"id":"1",
"prop":null,
"path":"1.parent",
"_childDocuments_":[
{
"id":"2",
"path":"2.parent.child"
}
]
}
It contains parent-child relationship structure denoted by _childDocuments_ key.
When I insert the documents in Solr via Post Tool, i.e., ./bin/post -c coreName data.json and query the Solr, then I get following response from Solr:
$ curl 'http://localhost:8983/solr/coreName/select?indent=on&q=*:*&wt=json'
{
"responseHeader":{
"status":0,
"QTime":0,
"params":{
"q":"*:*",
"indent":"on",
"wt":"json"}},
"response":{"numFound":1,"start":0,"docs":[
{
"id":"1",
"path":["1.parent"],
"_childDocuments_.id":[2],
"_childDocuments_.path":["2.parent.child"],
"_version_":1566718833663672320}]
}}
But when I try to insert the same JSON document via Index Handler, i.e., curl -
$ curl "http://localhost:8983/solr/coreName/update?commit=true" -H 'Content-type:application/json' --data-binary "#1.json"
{"responseHeader":{"status":400,"QTime":56},"error":{"metadata":["error-class","org.apache.solr.common.SolrException","root-error-class","org.apache.solr.common.SolrException"],"msg":"Unknown command 'id' at [11]","code":400}}
I get SolrException. But if I put the JSON in array, then it shows another error, i.e.,
[{
"id":"1",
"prop": null,
"path":"1.parent",
"_childDocuments_":[
{
"id":"2",
"path":"2.parent.child"
}
]
}]
Error:
$ curl 'http://localhost:8983/solr/coreName/update?commit=true' -H 'Content-type:application/json' --data "#/home/knoldus/practice/solr/1.json"
{"responseHeader":{"status":500,"QTime":3},"error":{"trace":"java.lang.NullPointerException\n\tat org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.mapValueClassesToFieldType(AddSchemaFieldsUpdateProcessorFactory.java:370)\n\tat org.apache.solr.update.processor.AddSchemaFieldsUpdateProcessorFactory$AddSchemaFieldsUpdateProcessor.processAdd(AddSchemaFieldsUpdateProcessorFactory.java:288)\n\tat org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)\n\tat org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)\n\tat org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)\n\tat org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)\n\tat org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)\n\tat org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)\n\tat org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)\n\tat org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)\n\tat org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)\n\tat org.apache.solr.update.processor.FieldNameMutatingUpdateProcessorFactory$1.processAdd(FieldNameMutatingUpdateProcessorFactory.java:74)\n\tat org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)\n\tat org.apache.solr.update.processor.FieldMutatingUpdateProcessor.processAdd(FieldMutatingUpdateProcessor.java:118)\n\tat org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:48)\n\tat org.apache.solr.update.processor.AbstractDefaultValueUpdateProcessorFactory$DefaultValueUpdateProcessor.processAdd(AbstractDefaultValueUpdateProcessorFactory.java:91)\n\tat org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.handleAdds(JsonLoader.java:492)\n\tat org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.processUpdate(JsonLoader.java:139)\n\tat org.apache.solr.handler.loader.JsonLoader$SingleThreadedJsonLoader.load(JsonLoader.java:115)\n\tat org.apache.solr.handler.loader.JsonLoader.load(JsonLoader.java:78)\n\tat org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:97)\n\tat org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:68)\n\tat org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:166)\n\tat org.apache.solr.core.SolrCore.execute(SolrCore.java:2306)\n\tat org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:658)\n\tat org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:464)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345)\n\tat org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:296)\n\tat org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)\n\tat org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)\n\tat org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)\n\tat org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)\n\tat org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)\n\tat org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)\n\tat org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)\n\tat org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)\n\tat org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)\n\tat org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)\n\tat org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)\n\tat org.eclipse.jetty.server.Server.handle(Server.java:534)\n\tat org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)\n\tat org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)\n\tat org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)\n\tat org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)\n\tat org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)\n\tat org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)\n\tat org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)\n\tat java.lang.Thread.run(Thread.java:745)\n","code":500}}
So, I have to remove "prop": null as well or make it an empty string, like this:
[{
"id":"1",
"prop": "",
"path":"1.parent",
"_childDocuments_":[
{
"id":"2",
"path":"2.parent.child"
}
]
}]
After making these modifications, when I insert the JSON doc. in Solr via curl, then it works fine.
$ curl 'http://localhost:8983/solr/coreName/update?commit=true' -H 'Content-type:application/json' --data "#/home/knoldus/practice/solr/1.json"
{"responseHeader":{"status":0,"QTime":851}}
And I get following response from Solr query:
$ curl 'http://localhost:8983/solr/coreName/select?indent=on&q=*:*&wt=json'
{
"responseHeader":{
"status":0,
"QTime":1,
"params":{
"q":"*:*",
"indent":"on",
"wt":"json"}},
"response":{"numFound":2,"start":0,"docs":[
{
"id":"2",
"path":["2.parent.child"]},
{
"id":"1",
"path":["1.parent"],
"_version_":1566719240059224064}]
}}
But here again I see a difference that _childDocuments_ have been indexed as separate documents.
So, I have following questions on two different methods of data indexing in Solr:
Why Post Tool ./bin/post does not index _childDocuments_ separately like Request Handler /update?
Why Request Handler /update requires JSON document to be wrapped in array?
And last, why Request Handler /update cannot handle null values whereas Post Tool can?
PostTool is just some small java utility that reads the input you give to it, and sends it to Solr. It is not running inside Solr. I have not looked but I am pretty sure:
it does not detect childDocuments as a special value and sends it just like any other nested json
as explained in the docs, /update requires the array, you can send individual json doc to /update/json/docs
PostTool is converting the null to either "" and just ignoring the field

Related Links

Highcharts jQuery.parseJSON with series
Predis not saving parts of JSON string
Problems with Jquery .get() jqXHR shows a valid return, but calls .fail()
Convert nested hash table recursively in Perl
Cannot figure out reason for JDOMException SAXBuilder
BigQuery's handling of missing fields and unknown/extra fields when importing JSON
REST Updating field that has a looked up value from the database
Sinatra: concerning filters, halt and body on 404 error
Kendo dataviz pan and zoom
Creating Gson object in JSP
SyntaxError: Failed to parse JSON body: Unexpected token o
Twitter json count limit?
spring 3 mvc json insert error
Is there an established representation for the difference between two JSONs?
How to JSON format an HTTP error response in webapp2
Unexpected token } at the end of json response

Categories

HOME
sharepoint
ember.js
google-cloud-bigtable
smartphone
label
computer-vision
windows-store-apps
ruby-on-rails-3
concourse
mainframe
couchdb-2.0
apple-numbers
webdav
adsense
gspread
xamarin-studio
pjsip
cakephp-2.9
android-source
jboss7.x
nuxt.js
symfony2-easyadmin
postmessage
spring-cloud-config
dendrogram
apache-karaf
primes
linq-to-sql
multicore
cvs2svn
statusbar
branch
metis
ui5
hana-studio
reactiveui
smart-mobile-studio
ifstream
gettext
broadleaf-commerce
socketpair
axis-labels
installshield-2012
equivalence
jquery-multidatespicker
soundjs
rule
intellij-idea-2016
android-maps-v2
pdftk
median
homekit
onmouseover
w3-total-cache
data-integration
uiviewpropertyanimator
volume
selenium-firefoxdriver
teiid
push-diffusion
tomcat5
freetype2
dbscan
dac
handlebars.java
acm
persistence.xml
enyo
affix
vips
matcaffe
uialertview
textkit
android-snackbar
paypal-nvp
mptcp
manjaro
twitter-rest-api
stackframe
word-2013
c++-actor-framework
orientation-changes
snmp4j
csplit
visual-c++-2010-express
bignum
.net-cf-3.5
argb
shortcuts
fieldset
maven-ear-plugin
separation-of-concerns
entity-framework-ctp5
flexicious
mysql-error-1045
objectbrowser
curljs
opengl-es-lighting
open-graph-beta
hadoop-plugins
yajl
economics
self-reference
document-library

Resources

Mobile Apps Dev
Database Users
javascript
java
csharp
php
android
MS Developer
developer works
python
ios
c
html
jquery
RDBMS discuss
Cloud Virtualization
Database Dev&Adm
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App