Uploaded image for project: 'eZ Publish / Platform'
  1. eZ Publish / Platform
  2. EZP-26141

Search API appears to conduct some form of 'OR' search not an 'AND' search

    Details

      Description

      Search relevancy does not work. It appears to conduct some form of 'OR' search not an 'AND' search. If "red apple" is search term, the 1st results should return red AND apple content.

      When using API and FullText Criterion on query we expect default sorting on solr to be able to sort by relevancy (score).

      Specifically searching for "Red Apple" should return items in following order:

      • Exact hits on "Red Apple"
      • Hits that contains both words
      • Hits that contains both works with fuzziness
      • Hits that contain one of them
      • Hits that contain one of them with fuzziness
      1. FullTextCommand.php
        2 kB
        Miguel das Neves Jacinto

        Issue Links

          Activity

          Łukasz Serwatka created issue -
          André Rømcke made changes -
          Field Original Value New Value
          Fix Version/s Customer request [ 11018 ]
          André Rømcke made changes -
          Status Open [ 1 ] Confirmed [ 10037 ]
          André Rømcke made changes -
          Status Confirmed [ 10037 ] InputQ [ 10001 ]
          André Rømcke made changes -
          Assignee Petar Spanja [ petar.spanja@ez.no ]
          Petar Spanja (Inactive) made changes -
          Status InputQ [ 10001 ] Development [ 3 ]
          André Rømcke made changes -
          Summary Search appears to conduct some form of 'OR' search not and 'AND' search Search appears to conduct some form of 'OR' search not an 'AND' search
          André Rømcke made changes -
          Description Search relevancy does not work. It appears to conduct some form of 'OR' search not and 'AND' search. If red apple is search term, the 1st results should return red AND apple content.

          Really want for launch:
          Extend the fields displayed in search result. Currently is 'Name' only. The challenge is that this search is used across multiple content types. For example: When looking for an article, you would want to see 'headline' but this would not be available for images etc.

          NOTE: Using "Red Apple" is not a solution - as this will only return results for articles with this string, not ones with the words anywhere in the content.
          Search relevancy does not work. It appears to conduct some form of 'OR' search not an 'AND' search. If red apple is search term, the 1st results should return red AND apple content.

          Really want for launch:
          Extend the fields displayed in search result. Currently is 'Name' only. The challenge is that this search is used across multiple content types. For example: When looking for an article, you would want to see 'headline' but this would not be available for images etc.

          NOTE: Using "Red Apple" is not a solution - as this will only return results for articles with this string, not ones with the words anywhere in the content.
          André Rømcke made changes -
          Description Search relevancy does not work. It appears to conduct some form of 'OR' search not an 'AND' search. If red apple is search term, the 1st results should return red AND apple content.

          Really want for launch:
          Extend the fields displayed in search result. Currently is 'Name' only. The challenge is that this search is used across multiple content types. For example: When looking for an article, you would want to see 'headline' but this would not be available for images etc.

          NOTE: Using "Red Apple" is not a solution - as this will only return results for articles with this string, not ones with the words anywhere in the content.
          Search relevancy does not work. It appears to conduct some form of 'OR' search not an 'AND' search. If "red apple" is search term, the 1st results should return red AND apple content.

          Really want for launch:
          Extend the fields displayed in search result. Currently is 'Name' only. The challenge is that this search is used across multiple content types. For example: When looking for an article, you would want to see 'headline' but this would not be available for images etc.

          NOTE: Using "Red Apple" is not a solution - as this will only return results for articles with this string, not ones with the words anywhere in the content.
          Show
          Petar Spanja (Inactive) added a comment - Pull requests: https://github.com/ezsystems/ezplatform-solr-search-engine/pull/63 https://github.com/ezsystems/ezpublish-kernel/pull/1770
          Show
          Petar Spanja (Inactive) added a comment - Fixed in ezplatform-solr-search-engine: 1.0: https://github.com/ezsystems/ezplatform-solr-search-engine/commit/566ffced49759c7d18edcc58a6159f6ad1ce613f master: https://github.com/ezsystems/ezplatform-solr-search-engine/commit/566ffced49759c7d18edcc58a6159f6ad1ce613f Tested in ezpublish-kernel: 6.3 and 6.4 https://github.com/ezsystems/ezplatform-solr-search-engine/commit/566ffced49759c7d18edcc58a6159f6ad1ce613f 6.5 and master: https://github.com/ezsystems/ezpublish-kernel/commit/bfe4d8637fb10e1a00a7b5598a4dce9e9a967571 https://github.com/ezsystems/ezpublish-kernel/commit/1520cef17decb24750f1c8904e17bf5427fe5731
          Petar Spanja (Inactive) made changes -
          Status Development [ 3 ] Development Review done [ 10028 ]
          Affects Version/s 1.6.0 [ 14600 ]
          Fix Version/s 1.4.2 [ 14595 ]
          Fix Version/s 1.5.1 [ 14598 ]
          Fix Version/s 1.3.3 [ 14599 ]
          Fix Version/s 1.6.0 [ 14600 ]
          Fix Version/s 5.4.9 [ 14605 ]
          Assignee Petar Spanja [ petar.spanja@ez.no ]
          Petar Spanja (Inactive) made changes -
          Status Development Review done [ 10028 ] Documentation Review done [ 10011 ]
          Miguel das Neves Jacinto (Inactive) made changes -
          Status Documentation Review done [ 10011 ] QA [ 10008 ]
          Hide
          Miguel das Neves Jacinto (Inactive) added a comment -

          I could not see any difference with the fix applied. In PlatformUI the search always return the same results in the same order.

          Having 3 article with titles 'Red', 'Red apples' and 'Apples' and searching for 'Red apples' always returns the 3 articles and the article 'Red' comes first since it was created first.

          Not sure if this is useful information, analyzing the response of the search the contents returned all have score 0 but using the same query on the solr backend all the contents have score 1.

          I could use some further information on this issue, since the supposed behavior is not really clear.

          Show
          Miguel das Neves Jacinto (Inactive) added a comment - I could not see any difference with the fix applied. In PlatformUI the search always return the same results in the same order. Having 3 article with titles 'Red', 'Red apples' and 'Apples' and searching for 'Red apples' always returns the 3 articles and the article 'Red' comes first since it was created first. Not sure if this is useful information, analyzing the response of the search the contents returned all have score 0 but using the same query on the solr backend all the contents have score 1. I could use some further information on this issue, since the supposed behavior is not really clear.
          Paulo Nunes (Inactive) made changes -
          Flagged Impediment [ 10000 ]
          Hide
          Paulo Nunes (Inactive) added a comment - - edited

          Just did a simple test as well, creating 3 articles:

          Article # #1 #2 #3
          Name red apple red apple
          Short Title red apple two three

          And when I search for "red apple", I have the three of them in the results list, meaning that an OR is being made and not an AND.

          Show
          Paulo Nunes (Inactive) added a comment - - edited Just did a simple test as well, creating 3 articles: Article # #1 #2 #3 Name red apple red apple Short Title red apple two three And when I search for "red apple", I have the three of them in the results list, meaning that an OR is being made and not an AND.
          Hide
          Miguel das Neves Jacinto (Inactive) added a comment -

          sending back to dev

          Show
          Miguel das Neves Jacinto (Inactive) added a comment - sending back to dev
          Miguel das Neves Jacinto (Inactive) made changes -
          Status QA [ 10008 ] InputQ [ 10001 ]
          Assignee Miguel das Neves Jacinto [ miguel.jacinto@ez.no ]
          André Rømcke made changes -
          Status InputQ [ 10001 ] Development [ 3 ]
          Assignee André Rømcke [ andre.romcke@ez.no ]
          André Rømcke made changes -
          Status Development [ 3 ] Backlog [ 10000 ]
          André Rømcke made changes -
          Status Backlog [ 10000 ] Development [ 3 ]
          André Rømcke made changes -
          Status Development [ 3 ] Development Review done [ 10028 ]
          Assignee André Rømcke [ andre.romcke@ez.no ]
          André Rømcke made changes -
          Status Development Review done [ 10028 ] Documentation Review done [ 10011 ]
          André Rømcke made changes -
          Description Search relevancy does not work. It appears to conduct some form of 'OR' search not an 'AND' search. If "red apple" is search term, the 1st results should return red AND apple content.

          Really want for launch:
          Extend the fields displayed in search result. Currently is 'Name' only. The challenge is that this search is used across multiple content types. For example: When looking for an article, you would want to see 'headline' but this would not be available for images etc.

          NOTE: Using "Red Apple" is not a solution - as this will only return results for articles with this string, not ones with the words anywhere in the content.
          Search relevancy does not work. It appears to conduct some form of 'OR' search not an 'AND' search. If "red apple" is search term, the 1st results should return red AND apple content.

          When using API and FullText Criterion on query we expect default sorting on solr to be able to sort by relevancy (score).

          Specifically searching for "Red Apple" should return items in following order:
          - Exact hits on "Red Apple"
          - Hits that contains both words
          - Hits that contains both works with fuzziness
          - Hits that contain one of them
          - Hits that contain one of them with fuzziness
           
          André Rømcke made changes -
          Link This issue is cloned by EZP-26325 [ EZP-26325 ]
          Miguel das Neves Jacinto (Inactive) made changes -
          Status Documentation Review done [ 10011 ] QA [ 10008 ]
          Paulo Nunes (Inactive) made changes -
          Flagged Impediment [ 10000 ]
          Hide
          Miguel das Neves Jacinto (Inactive) added a comment -

          ping André Rømcke, Petar Španja
          Testing using the API only always returned the same results, with and without the fix.

          I tested this issue using a Symfony Command (see attached) making a basic FullText search. There must be something I have missed because the result seem right even without the patch.
          Having the contents 'red', 'red apples' and 'apples' searching for 'red' returns(in this order) 'red', 'red apples' and searching for 'red apples' returns(in this order) 'red apples', 'red', 'apples'. This looks like the supposed but I am reproducing it without the fix and with the fix nothing changes.

          I attached the Command I made so you can take a look and see if I am missing some detail.

          Show
          Miguel das Neves Jacinto (Inactive) added a comment - ping André Rømcke , Petar Španja Testing using the API only always returned the same results, with and without the fix. I tested this issue using a Symfony Command (see attached) making a basic FullText search. There must be something I have missed because the result seem right even without the patch. Having the contents 'red', 'red apples' and 'apples' searching for 'red' returns(in this order) 'red', 'red apples' and searching for 'red apples' returns(in this order) 'red apples', 'red', 'apples'. This looks like the supposed but I am reproducing it without the fix and with the fix nothing changes. I attached the Command I made so you can take a look and see if I am missing some detail.
          Hide
          Petar Spanja (Inactive) added a comment -

          [~miguel.jacinto@ez.no] Is that with Solr or Legacy engine? The fix is only for Solr.

          Show
          Petar Spanja (Inactive) added a comment - [~miguel.jacinto@ez.no] Is that with Solr or Legacy engine? The fix is only for Solr.
          Miguel das Neves Jacinto (Inactive) made changes -
          Attachment FullTextCommand.php [ 27450 ]
          Hide
          Miguel das Neves Jacinto (Inactive) added a comment -

          Petar Španja it is with solr

          Show
          Miguel das Neves Jacinto (Inactive) added a comment - Petar Španja it is with solr
          Miguel das Neves Jacinto (Inactive) made changes -
          Status QA [ 10008 ] InputQ [ 10001 ]
          Assignee Miguel das Neves Jacinto [ miguel.jacinto@ez.no ]
          André Rømcke made changes -
          Summary Search appears to conduct some form of 'OR' search not an 'AND' search Search API appears to conduct some form of 'OR' search not an 'AND' search
          Hide
          Bertrand Dunogier added a comment -

          Can you guys please clarify the status of this, and close it whenever possible ? Stuff has been shipped with 1.5.1, and we must try to keep a clean changelog.

          Show
          Bertrand Dunogier added a comment - Can you guys please clarify the status of this, and close it whenever possible ? Stuff has been shipped with 1.5.1, and we must try to keep a clean changelog.
          Hide
          Petar Spanja (Inactive) added a comment -

          [~miguel.jacinto@ez.no], Paulo Nunes

          I don't know how to help here – I have no problems reproducing the bug and fix both with kernel and Admin UI.
          The command provided looks OK, but it doesn't show what exact search string is used or what exact Content is indexed.

          Show
          Petar Spanja (Inactive) added a comment - [~miguel.jacinto@ez.no] , Paulo Nunes I don't know how to help here – I have no problems reproducing the bug and fix both with kernel and Admin UI. The command provided looks OK, but it doesn't show what exact search string is used or what exact Content is indexed.
          André Rømcke made changes -
          Status InputQ [ 10001 ] Closed [ 6 ]
          Resolution Fixed [ 1 ]
          Nuno Oliveira (Inactive) made changes -
          Link This issue relates to EZS-1155 [ EZS-1155 ]
          Alex Schuster made changes -
          Workflow EZ* Development Workflow [ 99860 ] EZEE Development Workflow [ 125924 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Confirmed Confirmed
          20d 4h 11m 1 André Rømcke 22/Aug/16 3:55 PM
          Confirmed Confirmed InputQ InputQ
          4s 1 André Rømcke 22/Aug/16 3:55 PM
          InputQ InputQ Development Development
          14d 22h 52m 2 André Rømcke 16/Sep/16 11:01 AM
          Development Development Backlog Backlog
          3s 1 André Rømcke 16/Sep/16 11:01 AM
          Backlog Backlog Development Development
          3s 1 André Rømcke 16/Sep/16 11:01 AM
          Development Development Development Review done Development Review done
          8d 1h 39m 2 André Rømcke 16/Sep/16 11:01 AM
          Development Review done Development Review done Documentation Review done Documentation Review done
          3m 27s 2 André Rømcke 16/Sep/16 11:04 AM
          Documentation Review done Documentation Review done QA QA
          19h 7m 2 miguel.jacinto@ez.no 16/Sep/16 11:30 AM
          QA QA InputQ InputQ
          1d 6h 21m 2 miguel.jacinto@ez.no 16/Sep/16 5:58 PM
          InputQ InputQ Closed Closed
          4d 17h 51m 1 André Rømcke 21/Sep/16 11:50 AM

            People

            • Assignee:
              Unassigned
              Reporter:
              Łukasz Serwatka
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: