Get the most out of your Centmin Mod LEMP stack
Become a Member

Install Xenforo CentOS 7.x How To Install Elasticsearch 7.x for Xenforo

Discussion in 'Centmin Mod User Tutorials & Guides' started by Sunka, Mar 6, 2020.

  1. eva2000

    eva2000 Administrator Staff Member

    54,647
    12,230
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,799
    Local Time:
    5:16 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+
    you lock whatever elasticsearch packages that were installed only

     
  2. pamamolf

    pamamolf Premium Member Premium Member

    4,084
    428
    83
    May 31, 2014
    Ratings:
    +834
    Local Time:
    9:16 AM
    Nginx-1.25.x
    MariaDB 10.3.x
    On the net there are some info to use this:
    Code:
    yum versionlock elasticsearch
    Your recommendation if i got it correctly is to use it with the version and architecture as this is the package that i installed?
    Code:
    yum versionlock elasticsearch-7.6.2-1.x86_64
    or i need to versionlock any other related packages/dependencies to the above installation also?

    Confused......
     
    Last edited: Nov 18, 2020
  3. Itworx4me

    Itworx4me Active Member

    312
    28
    28
    Mar 14, 2017
    Ratings:
    +51
    Local Time:
    11:16 PM
    Nginx 1.25.X
    MariaDB 10.6.X
    Code (Text):
    cluster.name: CUSTOM NAME OF YOUR CLUSTER (same name "should" be inserted into your XenForo field in Admin Control Panel in field Elasticsearch Index Name)
    network.host: localhost
    http.port: 9200
    node.name: ANOTHER CUSTOM NAME
    cluster.initial_master_nodes: ANOTHER CUSTOM NAME (same as line above, and use all lowercase)

    What do people usually name there cluster. New to installing Elastic Search.

    Thanks,
    Itworx4me
     
  4. duderuud

    duderuud Premium Member Premium Member

    250
    82
    28
    Dec 5, 2020
    The Netherlands
    Ratings:
    +178
    Local Time:
    8:16 AM
    1.27 x
    10.6
    It really doesn't matter. I use an abbreviation of my website with the addition of "search" :)
     
  5. modder

    modder Member

    117
    16
    18
    Dec 6, 2019
    Ratings:
    +27
    Local Time:
    3:16 PM
  6. deltahf

    deltahf Premium Member Premium Member

    587
    265
    63
    Jun 8, 2014
    Ratings:
    +489
    Local Time:
    2:16 AM
    Thanks so much for this guide, @Sunka!

    One issue I ran into was not being able to start Elasticsearch:

    Code (Text):
    $ service elasticsearch start
    Starting elasticsearch (via systemctl):  Job for elasticsearch.service failed because the control process exited with error code. See "systemctl status elasticsearch.service" and "journalctl -xe" for details.
                                                               [FAILED]
    


    After further investigation in my logs (specifically, /var/log/elasticsearch/my_xenforo_index.log), I noticed there was a message related to possible file permissions errors. The problem turned out to be an issue with the permissions on the /var/lib/elasticsearch/nodes directory. Apparently removing ES6 removed the original "elasticsearch" user/group which owned that directory, so the permissions were not correct.

    This was easily fixed by simply updating the permissions on the directory:

    Code (Text):
    $ chown -R elasticsearch:elasticsearch /var/lib/elasticsearch/nodes
    


    Credit to this post on StackOverflow for originally mentioning this!

    Just for reference, this is how I deleted the index:

    Code (Text):
    $ curl -XDELETE localhost:9200/old_xenforo_index
    {"acknowledged":true}
    


    What does the --truncate option do? I would assume rebuilding the search index would "truncate" or remove the old index anyway?
     
    Last edited: Oct 28, 2021
  7. modder

    modder Member

    117
    16
    18
    Dec 6, 2019
    Ratings:
    +27
    Local Time:
    3:16 PM
    What's the recommended memory allocated for elasticsearch for a 1GB, 4GB and 8GB VPS respectively?
     
  8. rdan

    rdan Well-Known Member

    5,446
    1,408
    113
    May 25, 2014
    Ratings:
    +2,201
    Local Time:
    3:16 PM
    Mainline
    10.2
    Code:
    du -msh /var/lib/elasticsearch/
    
    Set -Xms & -Xmx
    That's I always follow, on how much data I stored.
     
  9. modder

    modder Member

    117
    16
    18
    Dec 6, 2019
    Ratings:
    +27
    Local Time:
    3:16 PM
    This is what I got for a freshly installed server (1GB RAM):
    Code (Text):
    [root@server ~]# du -msh /var/lib/elasticsearch/
    42M     /var/lib/elasticsearch/
    


    1) does it mean I'll need to set Xms and Xmx to 42MB?
    2) does this value (42M) grow on the complexity of the forum? So then I'll have to adjust it from time to time?
     
  10. rdan

    rdan Well-Known Member

    5,446
    1,408
    113
    May 25, 2014
    Ratings:
    +2,201
    Local Time:
    3:16 PM
    Mainline
    10.2
    Maybe x2 so you won't adjust it again for few months.
    Like 100M, not sure ES would start with that value.

    Yeah, as your forum will grow that size will grow also.
     
  11. deltahf

    deltahf Premium Member Premium Member

    587
    265
    63
    Jun 8, 2014
    Ratings:
    +489
    Local Time:
    2:16 AM
    For reference, my XF forum has 13.1 million posts and the size of that directory is 3.7GB for me.

    My -Xms and -Xmx values are both set to 16G, so I'm actually thinking that might be way too much...
     
  12. deltahf

    deltahf Premium Member Premium Member

    587
    265
    63
    Jun 8, 2014
    Ratings:
    +489
    Local Time:
    2:16 AM
    FYI, it is now recommended that you specify the JVM heap size in a separate file at /etc/elasticsearch/jvm.options.d/jvm.options:

    Code (Text):
    ################################################################
    ## IMPORTANT: JVM heap size
    ################################################################
    ##
    ## The heap size is automatically configured by Elasticsearch
    ## based on the available memory in your system and the roles
    ## each node is configured to fulfill. If specifying heap is
    ## required, it should be done through a file in jvm.options.d,
    ## and the min and max should be set to the same value. For
    ## example, to set the heap to 4 GB, create a new file in the
    ## jvm.options.d directory containing these lines:
    ##
    ## -Xms4g
    ## -Xmx4g
    ##
    ## See https://www.elastic.co/guide/en/elasticsearch/reference/7.16/heap-size.html
    ## for more information
    ##
    ################################################################
    


    Additional discussion at the XenForo community forums:

    Elasticsearch 7.11
     
  13. deltahf

    deltahf Premium Member Premium Member

    587
    265
    63
    Jun 8, 2014
    Ratings:
    +489
    Local Time:
    2:16 AM
    A security vulnerability in ElasticSearch has been discovered and the XenForo team has recommended this as a tentative patch for ElasticSearch versions 6.4 and up:

    PSA: Potential security vulnerability in Elasticsearch and more via Apache Log4j (Log4Shell)

    Add this to your custom configuration file in /etc/elasticsearch/jvm.options.d/jvm.options:

    Code (Text):
    -Dlog4j2.formatMsgNoLookups=true
    


    And restart ElasticSearch:

    Code (Text):
    service elasticsearch restart
    


    Also, if anyone is still running ES6, this would be a good reason to upgrade immediately.
     
    Last edited: Dec 14, 2021
  14. Itworx4me

    Itworx4me Active Member

    312
    28
    28
    Mar 14, 2017
    Ratings:
    +51
    Local Time:
    11:16 PM
    Nginx 1.25.X
    MariaDB 10.6.X
    Your pathway is incorrect. I think it should be /etc/elasticsearch/jvm.options.d/jvm.options:
     
  15. Itworx4me

    Itworx4me Active Member

    312
    28
    28
    Mar 14, 2017
    Ratings:
    +51
    Local Time:
    11:16 PM
    Nginx 1.25.X
    MariaDB 10.6.X
    If jvm.options isn't present in the jvm.options.d folder. Do I need to add it or am I doing something wrong?

    Thanks,
    Itworx4me
     
  16. deltahf

    deltahf Premium Member Premium Member

    587
    265
    63
    Jun 8, 2014
    Ratings:
    +489
    Local Time:
    2:16 AM
    Sorry for the typo, I edited my post. Yes, you will need to create the file yourself.
     
  17. deltahf

    deltahf Premium Member Premium Member

    587
    265
    63
    Jun 8, 2014
    Ratings:
    +489
    Local Time:
    2:16 AM
    ElasticSearch 7.16.1-1 is now available via yum update which patches the log4j1 vulnerability. Update immediately!

    Code (Text):
    yum --enablerepo=elasticsearch -y update elasticsearch
    


    Code (Text):
    $ curl -XGET 'localhost:9200'
    {
      "name" : "my_node",
      "cluster_name" : "my_cluster_name",
      "cluster_uuid" : "xxxxxxxxxx",
      "version" : {
        "number" : "7.16.1",
        "build_flavor" : "default",
        "build_type" : "rpm",
        "build_hash" : "5b38441b16b1ebb16a27c107a4c3865776e20c53",
        "build_date" : "2021-12-11T00:29:38.865893768Z",
        "build_snapshot" : false,
        "lucene_version" : "8.10.1",
        "minimum_wire_compatibility_version" : "6.8.0",
        "minimum_index_compatibility_version" : "6.0.0-beta1"
      },
      "tagline" : "You Know, for Search"
    }
    


    Official 7.16.1 release notes:

    Elasticsearch version 7.16.1 | Elasticsearch Guide [7.16] | Elastic
    • Disable JNDI lookups via the log4j2.formatMsgNoLookups system property #81622
    • Patch log4j jar to remove the JndiLookup class from the classpath #81629
     
  18. Itworx4me

    Itworx4me Active Member

    312
    28
    28
    Mar 14, 2017
    Ratings:
    +51
    Local Time:
    11:16 PM
    Nginx 1.25.X
    MariaDB 10.6.X
    I tried the yum update and its says there are no packages marked for update. :(

    I am currently on version 7.14.0
     
  19. pamamolf

    pamamolf Premium Member Premium Member

    4,084
    428
    83
    May 31, 2014
    Ratings:
    +834
    Local Time:
    9:16 AM
    Nginx-1.25.x
    MariaDB 10.3.x
    You can check also the log4j version using:

    Code:
    ls -lah /usr/share/elasticsearch/lib
     
  20. eva2000

    eva2000 Administrator Staff Member

    54,647
    12,230
    113
    May 24, 2014
    Brisbane, Australia
    Ratings:
    +18,799
    Local Time:
    5:16 PM
    Nginx 1.27.x
    MariaDB 10.x/11.4+