Blogs

  • Home
  • Blog
  • LogStack installation on CentOS7

LogStack installation on CentOS7

  • CentOS 7 64 bit with 4GB RAM
  • Elastic preinstalled on a machine
  • Unlock the ports for Elastic in the firewall:
  1. sudo firewall-cmd --zone=public --add-port=9200/tcp -permanent
  2. sudo firewall-cmd --zone=public --add-port=9300/tcp --permanent
  3. sudo firewall-cmd -reload

Key

Meaning of the colors in this guide:

  1. elasticsearch.repo → Text written in red is a free-form naming convention, whereas text written in black is not free-form.
  2. code examples appear in gray boxes
  3. an http link with curl command or import command is to be executed in the VM/Moba. Without curl in any web browser.
  4. this manual has CentOS7 preinstalled. The directories and commands may vary from OS to OS. In the case with the directory/command is shown as blue text, otherwise as orange text.

If not working on CentOS7, these links will help:

https://www.elastic.co/guide/en/elastic-stack-get-started/current/get-started-elastic-stack.html#install-elasticsearch

https://www.elastic.co/guide/en/logstash/7.6/dir-layout.html

https://www.elastic.co/guide/en/logstash/7.6/running-logstash.html

 

Step 1 - Install Java/OpenJDK

Java version 8 or 11 is required for the installation:

yum install java-1.8.0-openjdk-devel -y

 

Step 2 - Verify that was installed correctly

java -version

openjdk version "1.8.0_252"
OpenJDK Runtime Environment (build 1.8.0_252-b09)
OpenJDK 64-Bit Server VM (build 25.252-b09, mixed mode)

Step 3 - Creating the LogStack repo

First:

rpm --import https://artifacts.elastic.co/GPG-KEY-elasticsearch 

and create the repo

vi /etc/yum.repos.d/logstash.repo

Then copy the following into it:

[logstash-7.x]
name=Elastic repository for 7.x packages
baseurl=https://artifacts.elastic.co/packages/7.x/yum
gpgcheck=1
gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
enabled=1
autorefresh=1
type=rpm-md

 

Step 4 - Install and launch Logstack

The command to install is:

sudo yum install logstash -y

Start Logstack:

sudo systemctl enable logstash

sudo systemctl start logstash

 

Step 5 - Check if everything worked out

journalctl –unit logstash

localhost.localdomain systemd[1]: Started logstash.

 

Example - Simple Output

Note: From now on the IP 192.168.20.100 will be entered, this must always be adjusted to the respective own IP of the VM on which Elasticsearch is installed.

Execute the following command:

/usr/share/logstash/bin/logstash -e 'input { stdin { } } output {
elasticsearch { hosts => "192.168.20.100:9200" } }'

Now wait for the following output:

[Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}

After that you can just start typing. The output can be seen in the browser at http://192.168.20.100:9200/_search?pretty.

{
  "took" : 2,
  "timed_out" : false,
  "_shards" : {
    "total" : 2,
    "successful" : 2,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 9,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "logstash-2020.05.11-000001",
        "_type" : "_doc",
        "_id" : "-hDWAnIB5PIPaJRIY6Ll",
        "_score" : 1.0,
        "_source" : {
          "message" : "Hallo",
          "@timestamp" : "2020-05-11T08:25:03.941Z",
          "host" : "localhost.localdomain",
          "@version" : "1"
        }
      },

Example - Simple output to index

After creating an index logs in Elasticsearch, you can also move the output to the index by:

/usr/share/logstash/bin/logstash -e 'input { stdin { } } output { elasticsearch { hosts => "192.168.20.100:9200" index => "logs" } }'

Now wait for the following output:

[Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}

After that you can just start typing. The output can then be seen in the browser at http://192.168.20.100:9200/logs/_search?pretty

{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 2,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "logs",
        "_type" : "_doc",
        "_id" : "4usEA3IBWUPpcn4FpXBV",
        "_score" : 1.0,
        "_source" : {
          "host" : "localhost.localdomain",
          "@timestamp" : "2020-05-11T09:15:35.106Z",
          "@version" : "1",
          "message" : "Hi"
        }
      },

Example - Send file change to index

After creating an index logs in Elasticsearch, you can also move the output to the index by:

/usr/share/logstash/bin/logstash -e 'input { file { path => "/etc/resolv.conf"
start_position => beginning } } output { elasticsearch { hosts =>
"192.168.20.100:9200" index => "logs" } }'

Now wait for the following output:

[Api Webserver] agent - Successfully started Logstash API endpoint {:port=>9600}

Now you can change something in the folder /etc/resolv.conf and see the change
in the browser at http://192.168.20.100:9200/logs/_search?pretty.

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 16,
      "relation" : "eq"
    },
      {
        "_index" : "logs",
        "_type" : "_doc",
        "_id" : "VcKTA3IByzccBleDawQ9",
        "_score" : 1.0,
        "_source" : {
          "@version" : "1",
          "@timestamp" : "2020-05-11T11:51:32.291Z",
          "path" : "/etc/resolv.conf",
          "message" : "# Generated by NetworkManager Gentian Rrafshi",
          "host" : "localhost.localdomain"
        }

Example - Sending data changes via configfile in index

After creating an index logs in Elasticsearch, create a configfile elastic.conf in the subfolder /etc/logstash/conf.d/elastic.conf,
which looks like this:

eht:

input {
    file {
        path => "/etc/resolv.conf" 
        start_position => beginning
    }
}
output {
    elasticsearch {
        hosts => ["192.168.20.100:9200"]
        index => "logs"
   }
}

Test if the config fits:

sudo -u logstash /usr/share/logstash/bin/logstash --path.settings /etc/logstash -t

 

Now execute the following command:

/usr/share/logstash/bin/logstash -f elastic.conf --path.settings=/etc/logstash/

 

Now wait for the following output:

[logstasg.agent] Successfully started Logstash API endpoint {:port=>9600}

Now you can change something in the folder /etc/resolv.conf and save it.
The change can then be seen in the browser at http://192.168.20.100:9200/logs/_search?pretty


{
  "took" : 3,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 3,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "logs",
        "_type" : "_doc",
        "_id" : "bsITBHIByzccBleDHATP",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : "2020-05-11T14:11:00.834Z",
          "path" : "/etc/resolv.conf",
          "host" : "localhost.localdomain",
          "message" : "# Generated by Networkmanager Gentian Rrafshi",
          "@version" : "1"
        }

Important!

After each run systemctl restart logstash before!

You can display a list of all available indexes with this command: http://192.168.20.100:9200/_cat/indices

Check out our latest blogpost


Advanced Kubernetes Operators: Enhancing Automation and Management

At the heart of Kubernetes automation, Operators stand as the technical cornerstone, enhancing cluster efficiency and management through advanced operational mechanisms.

Any Questions?

Please feel free to contact us for any question that is not answered yet. 

We are looking forward to get in contact with you!

Newsletter

Design Escapes

KubeOps GmbH
Hinter Stöck 17
72406 Bisingen
Germany

  • Telefon:

    +49 7433 93724 90

  • Mail:

    This email address is being protected from spambots. You need JavaScript enabled to view it.

Download Area
Certified as

KubeOps GmbH is the owner of the Union trademark KubeOps with the registration number 018305184. 

© KubeOps GmbH. All rights reserved. Subsidiary of