Server error when requestion jpegPhoto attribute in REST search request

Description

When using REST search request, a server error is returned if jpegPhoto is listed inside attributes array.

Distribution Name and Version

# cat /etc/debian_version 
9.11

FusionDirectory Version

# dpkg -l | grep fusiondirectory
ii  fusiondirectory                          1.4-2~jenkinsbuild543                        all          Web Based LDAP Administration Program
ii  fusiondirectory-plugin-dsa               1.4-2~jenkinsbuild543                        all          dsa plugin for FusionDirectory
ii  fusiondirectory-plugin-dsa-schema        1.4-2~jenkinsbuild543                        all          LDAP schema for FusionDirectory dsa plugin
ii  fusiondirectory-plugin-mail              1.4-2~jenkinsbuild543                        all          base mail plugin for FusionDirectory
ii  fusiondirectory-plugin-mail-schema       1.4-2~jenkinsbuild543                        all          LDAP schema for FusionDirectory mail plugin
ii  fusiondirectory-plugin-personal          1.4-2~jenkinsbuild543                        all          Personal plugin for FusionDirectory
ii  fusiondirectory-plugin-personal-schema   1.4-2~jenkinsbuild543                        all          LDAP schema for FusionDirectory personal Plugin
ii  fusiondirectory-plugin-ppolicy           1.4-2~jenkinsbuild543                        all          ppolicy overlay module plugin for FusionDirectory
ii  fusiondirectory-plugin-ppolicy-schema    1.4-2~jenkinsbuild543                        all          LDAP schema for FusionDirectory ppolicy Plugin
ii  fusiondirectory-plugin-webservice        1.4-2~jenkinsbuild543                        all          webservice plugin for FusionDirectory
ii  fusiondirectory-plugin-webservice-schema 1.4-2~jenkinsbuild543                        all          schema for the webservice plugin for FusionDirectory
ii  fusiondirectory-schema                   1.4-2~jenkinsbuild543                        all          LDAP schema for FusionDirectory
ii  fusiondirectory-smarty3-acl-render       1.4-2~jenkinsbuild543                        all          Provide FusionDirectory ACL based rendering for Smarty3

Plugin with the defect

Webservice

PHP version used

# php -v
PHP 7.0.33-0+deb9u6 (cli) (built: Oct 24 2019 18:50:20) ( NTS )
Copyright (c) 1997-2017 The PHP Group
Zend Engine v3.0.0, Copyright (c) 1998-2017 Zend Technologies
    with Zend OPcache v7.0.33-0+deb9u6, Copyright (c) 1999-2017, by Zend Technologies

Origin of php packages

Debian

Steps to Reproduce

• Get a session token
• Do a search request with jpegPhoto attribute

curl -H "Session-Token: XXXX" 'https://fd.pfptnbdev.io/rest.php/v1/objects/user?base=uid=coudot,ou=users,dc=kroqi,dc=fr&scope=base&attrs[jpegPhoto]=1'

Expected behavior:

Response with jpegPhoto value

Actual behavior:

Error:

[{"message":"Error while encoding JSON result: Malformed UTF-8 characters, possibly incorrectly encoded","line":137,"file":"/usr/share/fusiondirectory/html/rest.php"}]

Reproduces how often:

100%

Additional Information

I don't have the issue when jpegPhoto is requested like this:

curl -H "Session-Token: XXXX" 'https://fd.pfptnbdev.io/rest.php/v1/objects/user/uid=coudot,ou=users,dc=kroqi,dc=fr/user/jpegPhoto'

I tried to set debug to max in FD, but nothing is printed in the logs. As this is the webservice, I can't use the IHM to see logs.

How can we debug a webservice call?

added Bugs plugin-webservice labels

assigned to @bmortier

assigned to @MCMic and unassigned @bmortier

changed due date to November 28, 2019

changed milestone to %FusionDirectory 1.4

added PJ1802-0188 label

I don't have the issue when jpegPhoto is requested like this

This is because using GET on a specific object will go through simpleplugin, and serialize data properly.

When searching objects, we get LDAP attributes, not FD attributes, basically. So the brute data for jpegPhoto is returned and it cannot be encoded to JSON properly because it is binary.

@coudot I see 3 potential solutions:

1. We catch Malformed UTF-8 characters and encode to base 64 in this case (problems: it means a same attribute may or may not be base64 encoded depending on its content)
2. We automatically encode to base64 LDAP attributes with specific syntax, in this case jpegPhoto is "1.3.6.1.4.1.1466.115.121.1.28", which is expected to require encoding: https://tools.ietf.org/html/rfc4517#section-3.3.17
3. We add an other possible value for attrs array, allowing the user to request b64 for an attribute, in this case you would request …&attrs[jpegPhoto]=b64 (and would get an array with b64 encoded values inside)

Solution 1 is close to what ldapsearch is doing, but ldapsearch shows '::' instead of ':' so that the user knows the value is b64 encoded. Not sure which kind of similar trick we could use here. Maybe return an array with [b64 => 'value'] or something like that.

My preference would be for solution 3.

mentioned in issue #6012 (closed)

mentioned in commit 44ad4892

I implemented the b64 option in fd#6060

added 1h of time spent at 2019-12-17

added To Be Tested label

assigned to @coudot and unassigned @MCMic

changed due date to January 20, 2020

Tested, it works, thanks!

added Added label and removed Bugs To Be Tested labels

removed due date

added user-manual label

removed user-manual label

closed