HTTP Client v2.4.0

This microservice allows performing HTTP requests and receive HTTP responses. It also can perform basic authentication

Configuration

Main configuration is done via setting following properties in a custom configuration:

• useTransport - use th2 transport or protobuf protocol to publish incoming/outgoing messages (true by default)
• maxBatchSize - max size of outgoing message batch (1000 by default)
• maxFlushTime - max message batch flush time (1000 by default)
• sessionAlias - session alias for incoming/outgoing TH2 messages (e.g. rest_api or null by default).
  This option is required when sessions option is missing in main config.
• sessions - map session alias to session configuration (empty by default)

Main / Sessions configuration

The options below can be specified either in the main block or in sessions blocks. When defined in main, they act as default values for all sessions, or as the parameters for a single session when sessionAlias is specified and sessions is not.

• https - enables HTTPS (false by default)
• host - host for HTTP requests (e.g. google.com)
• port - port for HTTP requests (80 by default or 443 if https = true)
• readTimeout - socket read timeout in ms (5000 by default)
• keepAliveTimeout - socket inactivity timeout in ms (15000 by default)
• validateCertificates - enables/disables server certificate validation (true by default)
• clientCertificate - path to client X.509 certificate in PEM format (requires certificatePrivateKey, null by default)
• certificatePrivateKey - path to client certificate RSA private key (PKCS8 encoded) in PEM format (null by default)
• defaultHeaders - map of default headers, and their values which will be applied to each request (existing headers are not affected, empty by default)
• auth - basic authentication settings (null by default)
• publishSentEvents - enables/disables publish of "message sent" events (true by default)

Authentication configuration

Basic authentication can be configured via setting following properties in the auth block of the main configuration

• username - basic authentication username
• password - basic authentication password

Configuration example

https: false
host: someapi.com
port: 334
sessionAlias: api_session
readTimeout: 5000
keepAliveTimeout: 15000
validateCertificates: true
clientCertificate: /path/to/certificate
certificatePrivateKey: /path/to/certificate/private/key
defaultHeaders:
  x-api-key: [ 'apikeywashere' ]
auth:
  username: coolusername
  password: verystrongpassword

MQ pins

input pins

• pin with subscribe, send and group attributes for requests using protobuf
• pin with subscribe, send and transport-group attributes for requests using th2 transport

Note: least one of pins above is required, it's mean that conn can handle messages via one or both protocols at the same time

output pins

Required output pins set depends on values of the useTransport option

useTransport option is true

• output queue with publish, transport-group attributes for transport group batch with responses and requests

useTransport option is false

• output queue with publish, raw attributes for transport group batch with responses and requests

Inputs/outputs

This section describes messages received and by produced by the service

Inputs

This service receives HTTP requests via MQ as transport message groups containing one of:

• a single raw message containing request body, which can have uri, method, and contentType properties in its metadata, which will be used in resulting request
• a single parsed message with Request message type containing HTTP request line and headers and a RawMessage described above

If both parsed and raw messages contain uri, method, and contentType, values from parsed message take precedence.
If none of them contain these values / and GET will be used as uri and method values respectively

Message descriptions

• Request

Field	Type	Description
method	String	HTTP method name (e.g. GET, POST, etc.)
uri	String	Request URI (e.g. /some/request/path?param1=value1&param2=value2...)
headers	List<Header>	HTTP headers (e.g. Host, Content-Length, etc.)

• Header

Field	Type	Description
name	String	HTTP header name
value	String	HTTP header value

Metadata properties prefixed with header- (case insensitive) are put in the HTTP request as headers with header- prefix removed

Outputs

HTTP requests and responses are sent via MQ as transport message groups containing a single raw message with a raw request or response.
raw message also has uri, method, and contentType metadata properties set equal to URI, method, and content type of request (or a response received for the request). In case of response request metadata properties and parent event id are copied into response message.

Deployment via infra-mgr

Here's an example of infra-mgr config required to deploy this service

apiVersion: th2.exactpro.com/v1
kind: Th2Box
metadata:
  name: http-client
spec:
  image-name: ghcr.io/th2-net/th2-conn-http-client
  image-version: 2.2.0-dev
  custom-config:
    https: false
    host: 127.0.0.1
    port: 8080
    sessionAlias: some_api
    readTimeout: 5000
    keepAliveTimeout: 15000
    validateCertificates: true
    clientCertificate: /secret/storage/cert.crt
    certificatePrivateKey: /secret/storage/private.key
    useTransport: false
    maxBatchSize: 1000
    maxFlushTime: 1000
    publishSentEvents: true
    defaultHeaders:
      x-api-key: [ 'apikeywashere' ]
    auth:
      username: user
      password: pwds
  type: th2-conn
  pins:
    - name: to_send_protobuf
      connection-type: mq
      attributes:
        - subscribe
        - send
        - group
    - name: to_send_transport
      connection-type: mq
      attributes:
        - subscribe
        - send
        - transport-group
    - name: out_group_request
      connection-type: mq
      attributes:
        - publish
        - raw
  extended-settings:
    service:
      enabled: false

Changelog

v2.4.0

• [GH-50] Attached message ids to send event
  • provided publishSentEvents option.
• [GH-44] Implemented multi-session feature
  • provided sessions option.
• Updated:
  • th2 gradle plugin: 0.3.14 (bom: 4.14.3)
  • kotlin: 2.3.0
  • kotlin-logging: 7.0.14
  • common: 5.17.1-dev

v2.3.2

• Produce multi-platform docker image
  • migrated to amazoncorretto:11-alpine-jdk docker image as base
• Updated:
  • th2 gradle plugin: 0.3.10 (bom: 4.14.2)
  • kotlin: 2.2.21

v2.3.1

• Updated:
  • th2 gradle plugin: 0.3.9 (bom: 4.14.1)
  • kotlin: 2.2.10
  • kotlin-logging: 7.0.13
  • common: 5.16.1-dev
  • common-utils: 2.4.0-dev
  • rawhttp-core: 2.6.0

v2.3.0

• Update common to 5.14.0-dev
• Update common utils to 2.2.3-dev
• Migrate to th2 gradle plugin 0.1.1

v2.2.1

• Fixed problem - connect reorders sequence numbers when sending messages in parallel mode by one session alias.

v2.2.0

• Puts unique th2-request-id property to metadata of request/response messages
• Merged v0.8.1

v2.1.0

• Supports th2 transport protocol

• Use event / message batcher

  • messages are batched by session group
  • events are batched by session scope

• owasp upgrade to 8.2.1

• th2-common upgrade to 5.3.0

v2.0.0

• books and pages support
• owasp plugin check
• kotlin upgrade to 1.6.21
• th2-common upgrade to 5.2.0
• th2-bom upgrade to 4.2.0

v0.8.1

Fixed:

• Fix problem with exceeding the limit for sockets to acquire when and exception is thrown during that process.

v0.8.0

• Vulnerability check pipeline step
• th2-common upgrade to 3.44.0
• th2-bom upgrade to 4.1.0

v0.8.1

Fixed:

• Fix problem with exceeding the limit for sockets to acquire when and exception is thrown during that process.

v0.8.0

• Vulnerability check pipeline step
• th2-common upgrade to 3.44.0
• th2-bom upgrade to 4.1.0

v0.7.2

Added:

• Publish Failed to handle message group event for each message in group with present parent event id

v0.7.1

Changed:

• Bump com.exactpro.th2:common dependency to 3.41.1

v0.7.0

Added:

• Ability to pass headers via message metadata

v0.6.0

Future:

• Ability to send multiple requests in parallel

v0.5.3

Fixed:

• Convert method from incoming message group to uppercase for better compatibility with some servers

v0.5.2

Fixed:

• fixed bug with not sending all requests with body

v0.5.1

Fixed:

• fixed bug with parent id loss inside client after sending request

v0.5.0

Added:

• publish "sent" events for outgoing messages (requests)

v0.4.0

Added:

• support for client-side certificate

v0.3.0

Added:

• option to disable server certificate validation

v0.2.0

Added:

• copying of metadata properties from request into response

v0.1.2

Fixed:

• fixed bug with 'name: name' headers

v0.1.1

Fixed:

• potential endless-loop if onStart handler tries to send something

v0.1.0

Added:

• start/stop API to internal HTTP client

Updated:

• dependencies

v0.0.15

Added:

• removal of socket in case of network error

v0.0.13

Changed:

• response message will be with same parent event id as request message batch

v0.0.10

Fixed:

• high idle CPU usage

v0.0.9

Fixed:

• expired sockets weren't removed from the cache

v0.0.8

Changed:

• use separate sequences for request and response message streams
• don't overwrite Content-Length header if it's already present

v0.0.7

Fixed:

• invalid error message in case of unexpected incoming message type