Certain debug inputs from channels now only visible in systemctl status output in DE 2.0rc2

[StevenCTimm]

There are some debug outputs from the Amazon sources which previously were going to standard out (or possibly standard error)
and now seem to be visible only via the systemctl status command
. since I believe that systemctl status can be run by a non-privileged user we need to do something about this.
Note that one of those outputs is an Amazon Secret Access Key which I've redacted.

Some of this may have also been going in on 1.7.

Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'date': 'Thu, 24 Mar 2022 11:58:47 GMT',
Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'x-amzn-requestid': '8bd107c7-d35e-44fe-a1ca-f68d3fd6c7c2'},
Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'HTTPStatusCode': 200,
Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'RequestId': '8bd107c7-d35e-44fe-a1ca-f68d3fd6c7c2',
Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'RetryAttempts': 0}}
Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: {'AssumedRoleUser': {'Arn': 'arn:aws:sts::159067897602:assumed-role/CalculateBill/rol...ssion',
Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'AssumedRoleId': 'AROAI3T6KYFHZFV7446KK:roleSwitchSession'},
Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'Credentials': {'AccessKeyId': 'ASIASKCJPP4BMLZZYAL2',
Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'Expiration': datetime.datetime(2022, 3, 24, 14, 0, 30, tzinfo=tzlocal()),
Mar 24 13:00:30 fermicloud467.fnal.gov decisionengine[4040]: 'SecretAccessKey': 'xxxxxxx',

[mambelli]

@vitodb could you check if the latest changes to master (#670) fixed this?

[StevenCTimm]

(This certainly was also going on in 1.7 it turns out, it's been happening for quite a while).

[goodenou]

I am curious about what the status is here. Has the problem been fixed?

[StevenCTimm]

Haven't tested a 2.0 series DE for a while but I'm not aware of any fix that has been done. just start up a 2.0 de and do de-client --status and see all the junk that spews out, especially if you have debug enabled. Certainly it is still happening in the 1.7.4 debug issue.

[goodenou]

The PR that (I believe) fixed these issues #670 was merged in on December 9th 2022, which is after the 1.7.4 and 2.0 DE tags were cut. Can we try running something that is dated post December 9th as a test?

[StevenCTimm]

The verbose output is still available in systemctl status as of DE 1.7.5
And I have verified it can be seen by a non-privileged user either running systemctl status
or journalctl.

[decisionengine@hepcsvc03 ~]$ journalctl _SYSTEMD_UNIT=decisionengine.service -S '2023-10-03'

[goodenou]

OK. I will look into this. Is it ONLY the Amazon sources, or are there other sources that exhibit this behavior?

[StevenCTimm]

There are others.. on hepcsvc03 you can do journalctl -xe and see the history of the other sources that are there.

[mambelli]

@StevenCTimm does #702 fix all the remaining ones or are there others?

[StevenCTimm]

We will only know once someone cuts a release that includes PR #702 which has still not been done.