The GUI and CLI are not the same.
The CLI tool needs little (if any) metadata and simply acts as a pass-through tool, ie, it passes your commands to ASE and then passes the results back to you (with some basic display formatting thrown in for good measure).
The GUI needs a good bit more metadata so that it knows how to process data being sent back from ASE. [NOTE: I don't use the GUI ... I'm an 'old' command line type of person ... so I don't have a lot of details on the inner workings of the GUI.]
The error you're receiving has to do with the GUI trying to obtain some metadata (that's what the sp_jdbc_primarykey stored proc is used for) prior to it running the sp_activeroles stored proc. The CLI does not need this metadata (ie, it does *not* call sp_jdbd_primarykey) so it doesn't generate an error. In neither case is the actual sp_activeroles stored proc invocation failing.
------------------
Since this is a new ASE 16 instance I'm wondering if perhaps some of the jdbc-related procs need to be manually loaded (though this may also depend on how you created the ASE 16 instance, too).
I'd suggest posting back with the complete output from running sp_version (via the CLI to eliminate errors).
In the meantime you could also try loading the latest/greatest set of jdbc/jConnect related stored procs.
On the host where ASE is running, under %SYBASE% you should see a directory named jConnect* (pick the one with the highest number in the name), then under the 'sp' subdirectory should be a collection of sql_server*sql scripts. Hopefully there's a version of the file with a '16' in the name.
Load the sql_server*16*sql script into your ASE 16 instance with the CLI, eg:
isql -S SYBASE1600 -U ... -P ... -i sqlserver*16*sql
Then try running your sp_activeroles query from the GUI.
NOTE: I don't have a copy of ASE 16 in front of me at the time so if in doubt about the above directions then post the output from the following:
=======================
-- from windows DOS prompt:
cd %SYBASE%
dir
-- for each jConn* directory, cd to jConn*/sp and get a list of files
cd jConnect...
cd sp
dir
=======================
------------------
Also ...
Where are you running the GUI from? The same host as the ASE 16 instance, or a different host?
Do you still get your errors if running the GUI from the host where ASE 16 is running?
[If running the GUI from an older Sybase software installation I'm wondering if there could be some compatibility issues with running the older GUI against ASE 16 ... hence the suggestion to try running the GUI that comes with ASE 16.]