Uploaded image for project: 'eZ Publish / Platform'
  1. eZ Publish / Platform
  2. EZP-26368

As a developer I want to index custom data with Solr Search Engine

    Details

      Description

      Solr Search Engine should provide a way for developers to index custom data to the Content or Location documents.

        Issue Links

          Activity

          Show
          Dominika Kurek added a comment - Documented in https://doc.ez.no/display/DEVELOPER/Solr+Bundle#SolrBundle-DocumentFieldMappers
          Hide
          Rui Silva added a comment -

          Hello Petar Spanja.
          So, I've tried following the documentation provided here to create my custom field mapper (I can provide my custom bundle with it, if needed), and I did the following, afterwards:
          Created a folder "Parent" under "eZ Platform", and under it, I created a folder "MyChild".
          I have Solr up and running and configured with autoSoftCommit to 100 (which I believe it has to be, if not please correct me).
          Then I access Solr admin interface and run the following query:

          q: Parent
          fl: *
          wt: json
          

          (all other fields set to default, i.e.: most of them empty)
          and I get this:

          {
            "responseHeader": {
              "status": 0,
              "QTime": 7,
              "params": {
                "fl": "*",
                "indent": "true",
                "q": "Parent",
                "_": "1480492048851",
                "wt": "json"
              }
            },
            "response": {
              "numFound": 1,
              "start": 0,
              "docs": [
                {
                  "id": "content67langenggb",
                  "content_id": "67",
                  "document_type_id": "content",
                  "type_id": "1",
                  "version_no_i": 1,
                  "status_id": "1",
                  "name_s": "Parent",
                  "creator_id": "14",
                  "owner_id": "14",
                  "owner_user_group_mid": [
                    "4",
                    "12",
                    "14"
                  ],
                  "section_id": "1",
                  "section_identifier_id": "standard",
                  "section_name_s": "Standard",
                  "remote_id_id": "c45de5a18af317e636743a3c100009cb",
                  "modified_dt": "2016-11-23T09:21:40Z",
                  "published_dt": "2016-11-23T09:21:40Z",
                  "language_code_ms": [
                    "eng-GB"
                  ],
                  "main_language_code_s": "eng-GB",
                  "always_available_b": true,
                  "location_visible_b": true,
                  "location_id_mid": [
                    "69"
                  ],
                  "location_parent_id_mid": [
                    "2"
                  ],
                  "location_remote_id_mid": [
                    "9664ff8b0c75655b47519d59f0564cb6"
                  ],
                  "location_path_string_mid": [
                    "/1/2/69/"
                  ],
                  "main_location_id": "69",
                  "main_location_parent_id": "2",
                  "main_location_remote_id_id": "9664ff8b0c75655b47519d59f0564cb6",
                  "main_location_visible_b": true,
                  "main_location_path_id": "/1/2/69/",
                  "main_location_depth_i": 2,
                  "main_location_priority_i": 0,
                  "group_mid": [
                    "1"
                  ],
                  "object_state_mid": [
                    "1"
                  ],
                  "folder_name_value_s": "Parent",
                  "folder_short_description_value_s": "",
                  "folder_description_value_s": "",
                  "meta_indexed_language_code_s": "eng-GB",
                  "meta_indexed_is_main_translation_b": true,
                  "meta_indexed_is_main_translation_and_always_available_b": true,
                  "_version_": 1551780178759254000,
                  "_root_": "content67langenggb",
                  "timestamp": "2016-11-23T09:21:41.109Z"
                }
              ]
            }
          }
          

          So, I see my Parent content, but not the MyChild content in the xml.
          As far as I believe, and after our chat on Skype, that was the impression I was under that should happen.
          Can you confirm, or otherwise correct any wrong step I may have, and any assumption or misinterpretation of the results I'm to expect?
          Thanks in advance.

          Show
          Rui Silva added a comment - Hello Petar Spanja . So, I've tried following the documentation provided here to create my custom field mapper (I can provide my custom bundle with it, if needed), and I did the following, afterwards: Created a folder "Parent" under "eZ Platform", and under it, I created a folder "MyChild". I have Solr up and running and configured with autoSoftCommit to 100 (which I believe it has to be, if not please correct me). Then I access Solr admin interface and run the following query: q: Parent fl: * wt: json (all other fields set to default, i.e.: most of them empty) and I get this: { "responseHeader": { "status": 0, "QTime": 7, "params": { "fl": "*", "indent": "true", "q": "Parent", "_": "1480492048851", "wt": "json" } }, "response": { "numFound": 1, "start": 0, "docs": [ { "id": "content67langenggb", "content_id": "67", "document_type_id": "content", "type_id": "1", "version_no_i": 1, "status_id": "1", "name_s": "Parent", "creator_id": "14", "owner_id": "14", "owner_user_group_mid": [ "4", "12", "14" ], "section_id": "1", "section_identifier_id": "standard", "section_name_s": "Standard", "remote_id_id": "c45de5a18af317e636743a3c100009cb", "modified_dt": "2016-11-23T09:21:40Z", "published_dt": "2016-11-23T09:21:40Z", "language_code_ms": [ "eng-GB" ], "main_language_code_s": "eng-GB", "always_available_b": true, "location_visible_b": true, "location_id_mid": [ "69" ], "location_parent_id_mid": [ "2" ], "location_remote_id_mid": [ "9664ff8b0c75655b47519d59f0564cb6" ], "location_path_string_mid": [ "/1/2/69/" ], "main_location_id": "69", "main_location_parent_id": "2", "main_location_remote_id_id": "9664ff8b0c75655b47519d59f0564cb6", "main_location_visible_b": true, "main_location_path_id": "/1/2/69/", "main_location_depth_i": 2, "main_location_priority_i": 0, "group_mid": [ "1" ], "object_state_mid": [ "1" ], "folder_name_value_s": "Parent", "folder_short_description_value_s": "", "folder_description_value_s": "", "meta_indexed_language_code_s": "eng-GB", "meta_indexed_is_main_translation_b": true, "meta_indexed_is_main_translation_and_always_available_b": true, "_version_": 1551780178759254000, "_root_": "content67langenggb", "timestamp": "2016-11-23T09:21:41.109Z" } ] } } So, I see my Parent content, but not the MyChild content in the xml. As far as I believe, and after our chat on Skype, that was the impression I was under that should happen. Can you confirm, or otherwise correct any wrong step I may have, and any assumption or misinterpretation of the results I'm to expect? Thanks in advance.
          Hide
          Petar Spanja added a comment -

          Rui Silva please do provide the mapper code.
          You can also try to run indexing command (or otherwise force Solr to commit), just to eliminate that part.

          Show
          Petar Spanja added a comment - Rui Silva please do provide the mapper code. You can also try to run indexing command (or otherwise force Solr to commit), just to eliminate that part.
          Hide
          Rui Silva added a comment -

          Petar Spanja, it is no longer necessary, thank you.
          I reinstalled platform and did everything from scratch, and it is now working as expected.

          Show
          Rui Silva added a comment - Petar Spanja , it is no longer necessary, thank you. I reinstalled platform and did everything from scratch, and it is now working as expected.
          Hide
          Rui Silva added a comment -

          Tested and approved by QA for master.
          Although, I'd just make small notes on the documentation, such as:

          • "You can create you field mapper class anywhere you want inside your bundle, as long as when you register it as a service, the "class" parameter" on your services.yml matches the correct path."
            I'm advising towards this since, when I started following the tutorial, I didn't understand where I'd need to create my mapper class (i.e.: if there was supposed to exist some default path structure or not, under which to create it), since only at the end, it says we'll register it as a service and provide a custom path for it.
          • Another thing: the concept of "block documents" does not seem precise and defined anywhere to me, as nowhere it seems to provide an accurate description of what it exactly is and what it represents.
            It refers in there about "Content Blocks" and "Documents", but I do not think this is well explained for a section that stated firsthand "(...) you first need to understand how the data is indexed with Solr Search engine", but where the concepts are pretty vague..
          • Also, a small nitpick, the Heading level for the title "Document Field Mappers" is bigger than the other son the same doc page.

          Anyways, for a developer's perspective, this does not affect one's skill on following up the tutorial and accomplishing the purpose, so I'll validate the doc as well, and if later on, if anyone might agree with my POVs, you may change it or not at will.

          Show
          Rui Silva added a comment - Tested and approved by QA for master. Although, I'd just make small notes on the documentation, such as: "You can create you field mapper class anywhere you want inside your bundle, as long as when you register it as a service, the "class" parameter" on your services.yml matches the correct path." I'm advising towards this since, when I started following the tutorial, I didn't understand where I'd need to create my mapper class (i.e.: if there was supposed to exist some default path structure or not, under which to create it), since only at the end, it says we'll register it as a service and provide a custom path for it. Another thing: the concept of "block documents" does not seem precise and defined anywhere to me, as nowhere it seems to provide an accurate description of what it exactly is and what it represents. It refers in there about "Content Blocks" and "Documents", but I do not think this is well explained for a section that stated firsthand "(...) you first need to understand how the data is indexed with Solr Search engine", but where the concepts are pretty vague.. Also, a small nitpick, the Heading level for the title "Document Field Mappers" is bigger than the other son the same doc page. Anyways, for a developer's perspective, this does not affect one's skill on following up the tutorial and accomplishing the purpose, so I'll validate the doc as well, and if later on, if anyone might agree with my POVs, you may change it or not at will.

            People

            • Assignee:
              Unassigned
              Reporter:
              Petar Spanja
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: