Uploaded image for project: 'eZ Publish / Platform'
  1. eZ Publish / Platform
  2. EZP-21906

Search API returns wrong number of total objects with content having multiple locations

    Details

      Description

      When running a query with Search API, if one of the returned content objects has multiple locations, SearchResult->totalCount is wrong. It's superior to the actual number of results.
      Even more, when using LocationPriority sort clause, totalCount is even greater.

      Script to reproduce

        Issue Links

          Activity

          Hide
          Jérôme Vieilledent (Inactive) added a comment -

          Cannot reproduce this issue. Tested on master and stable-5.1.
          Script used: https://gist.github.com/lolautruche/27db1c1d2999c0e4554f

          Show
          Jérôme Vieilledent (Inactive) added a comment - Cannot reproduce this issue. Tested on master and stable-5.1. Script used: https://gist.github.com/lolautruche/27db1c1d2999c0e4554f
          Hide
          Jérôme Vieilledent (Inactive) added a comment -

          Paulo Bras: Could it be related to EZP-21700 ? In your tests, do you have several translations ? This would explain the gap between totalCount and actual result count. totalCount would be actually right, but searchHits would have duplicates.

          Show
          Jérôme Vieilledent (Inactive) added a comment - Paulo Bras : Could it be related to EZP-21700 ? In your tests, do you have several translations ? This would explain the gap between totalCount and actual result count. totalCount would be actually right, but searchHits would have duplicates.
          Hide
          Paulo Bras (Inactive) added a comment -

          @Jérome, i saw that one a long time ago. my case does not need multiple translations. also, the other issue is about the language criterion. i did not include that criterion in the example, and made sure that no multiple translated objects were used for the test.

          Show
          Paulo Bras (Inactive) added a comment - @Jérome, i saw that one a long time ago. my case does not need multiple translations. also, the other issue is about the language criterion. i did not include that criterion in the example, and made sure that no multiple translated objects were used for the test.
          Hide
          Paulo Bras (Inactive) added a comment - - edited

          @ Jérome:

          tested with your script command and mine to compare. the test case is a random folder already present in system, with two articles. default roles, permissions, standard section, same language for all objects. one article, the "mehasdrubal" has a secondary location outside the test folder
          the result is:

          your script, hardcoded to node 1159:

          Query #1, not using sort
          Total count: 2 / Effective count: 2
          Query #2, using ContentName sort clause
          Total count: 2 / Effective count: 2
          Query #3, using LocationPriority sort clause
          Total count: 3 / Effective count: 2
          

          -----------------
          my script, no sort clauses( test:newsearch '1159' 0 0 0 - each flag activates a sort clause):

          TotalCount :2 items;count in array:2 items
          object found:cache hidden
          LOCATIONS
            /1/2/1159/1160/  (/closed-folder/cache-hidden)
          object found:mehasdrubal
          LOCATIONS
            /1/2/125/  (/mehasdrubal)
            /1/2/1159/1166/  (/closed-folder/mehasdrubal)
          

          -----------------
          1 sort clause active ( test:newsearch '1159' 1 0 0 ):

          locationPriority clause  active
          TotalCount :3 items;count in array:2 items
          object found:cache hidden
          LOCATIONS
            /1/2/1159/1160/  (/closed-folder/cache-hidden)
          object found:mehasdrubal
          LOCATIONS
            /1/2/125/  (/mehasdrubal)
            /1/2/1159/1166/  (/closed-folder/mehasdrubal)
          

          --------------------
          2 sort clauses ( test:newsearch '1159' 1 1 0) :

          locationPriority clause  active
          field clause active
          TotalCount :3 items;count in array:2 items
          object found:cache hidden
          LOCATIONS
            /1/2/1159/1160/  (/closed-folder/cache-hidden)
          object found:mehasdrubal
          LOCATIONS
            /1/2/125/  (/mehasdrubal)
            /1/2/1159/1166/  (/closed-folder/mehasdrubal)
          

          my command shows on each run the equivalent of each of your queries, just shows extra details that were used to show as examples to the client.

          both commands without sortclauses return the same, and correct values for the totalCount. as sortclauses are added, the results start to deviate, and in both commands the result is wrong, the totalCount varies from the manual count(). so, it appears that the number of totalCount depends on the number of sortclauses AND which clause is used. ezFind is mentioned at the start of the project issue, since client has it installed and it was investigated. but that subject was dropped early in the investigation, after being discarded as a cause.

          also, my script has a extra flag that shows the results wising offsetting on the query. together with the totalCount, it provides wrong results for pagination. basically, the client was getting it all wrong because his results for pagination were based on the totalCount, which for larger query results (say, 20 results with a limit/offset of 5) starts to give wrong number for pagination.

          EDIT: "wsing" is not english, but "using" is

          Show
          Paulo Bras (Inactive) added a comment - - edited @ Jérome: tested with your script command and mine to compare. the test case is a random folder already present in system, with two articles. default roles, permissions, standard section, same language for all objects. one article, the "mehasdrubal" has a secondary location outside the test folder the result is: your script, hardcoded to node 1159: Query #1, not using sort Total count: 2 / Effective count: 2 Query #2, using ContentName sort clause Total count: 2 / Effective count: 2 Query #3, using LocationPriority sort clause Total count: 3 / Effective count: 2 ----------------- my script, no sort clauses( test:newsearch '1159' 0 0 0 - each flag activates a sort clause): TotalCount :2 items;count in array:2 items object found:cache hidden LOCATIONS /1/2/1159/1160/ (/closed-folder/cache-hidden) object found:mehasdrubal LOCATIONS /1/2/125/ (/mehasdrubal) /1/2/1159/1166/ (/closed-folder/mehasdrubal) ----------------- 1 sort clause active ( test:newsearch '1159' 1 0 0 ): locationPriority clause active TotalCount :3 items;count in array:2 items object found:cache hidden LOCATIONS /1/2/1159/1160/ (/closed-folder/cache-hidden) object found:mehasdrubal LOCATIONS /1/2/125/ (/mehasdrubal) /1/2/1159/1166/ (/closed-folder/mehasdrubal) -------------------- 2 sort clauses ( test:newsearch '1159' 1 1 0) : locationPriority clause active field clause active TotalCount :3 items;count in array:2 items object found:cache hidden LOCATIONS /1/2/1159/1160/ (/closed-folder/cache-hidden) object found:mehasdrubal LOCATIONS /1/2/125/ (/mehasdrubal) /1/2/1159/1166/ (/closed-folder/mehasdrubal) my command shows on each run the equivalent of each of your queries, just shows extra details that were used to show as examples to the client. both commands without sortclauses return the same, and correct values for the totalCount. as sortclauses are added, the results start to deviate, and in both commands the result is wrong, the totalCount varies from the manual count(). so, it appears that the number of totalCount depends on the number of sortclauses AND which clause is used. ezFind is mentioned at the start of the project issue, since client has it installed and it was investigated. but that subject was dropped early in the investigation, after being discarded as a cause. also, my script has a extra flag that shows the results wising offsetting on the query. together with the totalCount, it provides wrong results for pagination. basically, the client was getting it all wrong because his results for pagination were based on the totalCount, which for larger query results (say, 20 results with a limit/offset of 5) starts to give wrong number for pagination. EDIT: "wsing" is not english, but "using" is
          Hide
          Jérôme Vieilledent (Inactive) added a comment -

          OK, With this explanation, I was able to reproduce the issue.

          Show
          Jérôme Vieilledent (Inactive) added a comment - OK, With this explanation, I was able to reproduce the issue.
          Show
          Jérôme Vieilledent (Inactive) added a comment - PR: https://github.com/ezsystems/ezpublish-kernel/pull/640
          Show
          Jérôme Vieilledent (Inactive) added a comment - Fixed in master: https://github.com/ezsystems/ezpublish-kernel/commit/acf9e5f1a32cfa2e74edf8b83a9b184c67e21461
          Hide
          Pedro Resende (Inactive) added a comment -

          Tested and approved by Q.A.

          Show
          Pedro Resende (Inactive) added a comment - Tested and approved by Q.A.

            People

            • Assignee:
              Unassigned
              Reporter:
              Paulo Bras (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 0 minutes
                0m
                Remaining:
                Remaining Estimate - 0 minutes
                0m
                Logged:
                Time Spent - 2 days, 3 hours, 45 minutes
                2d 3h 45m