1.0.0 gem not available on rubygem.org

[onet-git]

Version 1.0.0 of this code has not been release to rubygem.org, which is breaking the fluent-plugin-relp CI build and preventing it from building and releasing a new version of that gem automatically. See https://travis-ci.org/ViaQ/fluent-plugin-relp.

It would be nice if this project also had CI setup, but that is a separate issue.

[onet-git]

@richm I am not sure why travis-ci was not setup for this project.

[richm]

@richm I am not sure why travis-ci was not setup for this project.

I don't know either. Please feel free to contribute support.

[richm]

@onet-git Would you be willing and able to take over maintenance of relp and fluent-plugin-relp? Basically, fork them to your repo, and I will deprecate the ViaQ versions. We (Red Hat) abandonded these years ago.

[onet-git]

What has Redhat chosen to pursue as the way to get logs from the syslog reliably into cloud base log system?

The fundamental issue is that fluentd still advertises these plugins as working and a "supported" solution. If fluentd and these plugins are the wrong tool, what is the right tool?

[richm]

What has Redhat chosen to pursue as the way to get logs from the syslog reliably into cloud base log system?

Nothing?

by syslog do you mean rsyslog?

what protocols does cloud base support?

The fundamental issue is that fluentd still advertises

where? links?

these plugins as working and a "supported" solution. If fluentd and these plugins are the wrong tool, what is the right tool?

I don't know, but this particular relp plugin never actually worked very well, and was not actively worked on except for a brief period of time.

[onet-git]

What has Redhat chosen to pursue as the way to get logs from the syslog reliably into cloud base log system?

Nothing?

by syslog do you mean rsyslog?

I mean rsyslog, i know that syslog-ng has its own reliable transport protocol. What I was referring to was ways to get logs from rsyslog into various cloud backends like ELK, loggly, splunk, etc.

fluentd was highly recommended to me by several individuals. Specifically as a parsing and forwarding layer because we don't want the machines generating the logs to directly talk to the cloud backends. This is both a security and layering concern (we don't want to update the logging config on every machine if we switch backends from say ELK to splunk).

what protocols does cloud base support?

The fundamental issue is that fluentd still advertises

where? links?

https://www.fluentd.org/plugins

these plugins as working and a "supported" solution. If fluentd and these plugins are the wrong tool, what is the right tool?

I don't know, but this particular relp plugin never actually worked very well, and was not actively worked on except for a brief period of time.

The plugin works now. Yet I also have open two pull requests to fix or improve the syslog support in fluentd and several more in the works. This makes me wonder if I am using the correct tool.

Since Redhat also explored this path and ended up abandoning it I am wondering what solution ended up working for you. Assuming you ever tried to solve a similar problem.

[richm]

What has Redhat chosen to pursue as the way to get logs from the syslog reliably into cloud base log system?

Nothing?
by syslog do you mean rsyslog?

I mean rsyslog, i know that syslog-ng has its own reliable transport protocol. What I was referring to was ways to get logs from rsyslog into various cloud backends like ELK, loggly, splunk, etc.

The RHEL rsyslog supports the syslog protocol (rfc3164 and rfc5424, which I think can use TLS, and queuing options to make it more reliable), as well as kafka and elasticsearch (not sure how reliable you would consider those to be). Fluentd can also receive syslog protocol, but I suppose that isn't as reliable as RELP.

fluentd was highly recommended to me by several individuals. Specifically as a parsing and forwarding layer because we don't want the machines generating the logs to directly talk to the cloud backends. This is both a security and layering concern (we don't want to update the logging config on every machine if we switch backends from say ELK to splunk).

what protocols does cloud base support?

The fundamental issue is that fluentd still advertises

where? links?

https://www.fluentd.org/plugins

these plugins as working and a "supported" solution. If fluentd and these plugins are the wrong tool, what is the right tool?

I don't know, but this particular relp plugin never actually worked very well, and was not actively worked on except for a brief period of time.

The plugin works now. Yet I also have open two pull requests to fix or improve the syslog support in fluentd and several more in the works. This makes me wonder if I am using the correct tool.

Red Hat doesn't use this relp plugin, but I would be happy to merge patches and release new versions of this plugin.

Since Redhat also explored this path and ended up abandoning it I am wondering what solution ended up working for you. Assuming you ever tried to solve a similar problem.

We have mostly been focused on the elasticsearch and syslog outputs of rsyslog, when we need to send logs from rsyslog to something other than another rsyslog (which we would use RELP for). The kafka support has been there for a while, but my teams have not used it.