google code pretty

顯示具有 Work 標籤的文章。 顯示所有文章
顯示具有 Work 標籤的文章。 顯示所有文章

2020-11-05

CentOS 8 x hadoop 2.7.3 NameNode HA Configuration

core-site.xml
<property>
	<name>fs.defaultFS</name>
	<value>hdfs://my-cluster-name</value>
</property>

<property>
	<name>ha.zookeeper.quorum</name>
	<value>host1:2181,host2:2181,host3:2181</value>
</property>

<property>
	<name>ipc.client.connect.max.retries</name>
	<value>100</value>
	<description> Indicates the number of retries a client will make to establisha server connection. </description>
</property>

<property>
	<name>ipc.client.connect.retry.interval</name>
	<value>10000</value>
	<description> Indicates the number of milliseconds a client will wait for before retrying to establish a server connection. </description>
</property>
hdfs-site.xml
<property>  
	<name>dfs.nameservices</name>  
	<value>my-cluster-name</value>  
</property>  

<property>  
	<name>dfs.ha.namenodes.my-cluster-name</name>  
	<value>host1,host2</value>  
</property>  

<property>
	<name>dfs.namenode.rpc-address.my-cluster-name.eyequila08</name>
	<value>host1:9000</value>
</property>

<property>
	<name>dfs.namenode.rpc-address.my-cluster-name.eyequila09</name>
	<value>host2:9000</value>
</property>

<property>
	<name>dfs.namenode.http-address.my-cluster-name.eyequila08</name>
	<value>host1:50070</value>
</property>

<property>
	<name>dfs.namenode.http-address.my-cluster-name.eyequila09</name>
	<value>host2:50070</value>
</property>

<property>
	<name>dfs.namenode.shared.edits.dir</name>
	<value>qjournal://host1:8485;host2:8485/eyequila</value>
</property>

<property>
	<name>dfs.client.failover.proxy.provider.my-cluster-name</name>
	<value>org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider</value>
</property>

<property>  
	<name>dfs.journalnode.edits.dir</name>  
	<value>/path/to/save/journal</value>  
</property>

<property>
	<name>dfs.qjournal.write-txns.timeout.ms</name>
	<value>600000000</value>
</property>

<property>  
	<name>dfs.ha.automatic-failover.enabled</name>  
	<value>true</value>  
</property>

<property>  
	<name>dfs.ha.fencing.methods</name>  
	<value>sshfence
	shell(/bin/true)
	</value>  
</property> 

<property>  
	<name>dfs.ha.fencing.ssh.private-key-files</name>  
	<value>/home/user_name/.ssh/id_rsa</value>  
</property>  

2020-03-25

Centos7 手動安裝 Openssl 1.1.1

先隨手紀錄一下(看了N個教學沒一個能一次成功的,最後是合併了兩個不同的教學步驟才成功)

在Centos7.6下編譯安裝openssl 1.1.1b
官網 https://www.openssl.org/source/ 下載 openssl-1.1.1b.tar.gz

Centos7.6 安裝編譯環境
# yum groupinstall "Development Tools"

解壓縮 開始編譯
# tar xf openssl-1.1.1b.tar.gz
# cd openssl-1.1.1b/
# ./config --prefix=/usr/local/openssl --openssldir=/usr/local/ssl
# make -j 2
# make install

導出庫文件
# echo  /usr/local/openssl/lib >> /etc/ld.so.conf.d/openssl.conf
# ldconfig
# "檢測版本信息"
# /usr/local/openssl/bin/openssl  version -a

設定環境變數
# vi /etc/ld.so.conf
加入: /usr/local/openssl/lib/
ldconfig -v
## 設定 openssl link
ln -s /usr/local/openssl/bin/openssl  /usr/local/bin/openssl
## 驗證是否安裝成功
openssl version -a

2018-07-06

VMware Workstation x Win7 x CentOS Linux 網路環境設定

當初弄這環境設定搞了半天才搞定,得把他記錄下來!

VM的網路連接有三種方式,分別是 Bridged, NAT和 host only三種模式,以我的理解用一句話來說的話就是

  • Bridged:自己是台獨立的機器,有自己的IP可以跟區網中的其他台電腦溝通,印象中是需要有自己的網卡才能這樣設定
  • NAT:就是透過NAT(Network Address Transformation)技術,讓VM藉由host主機來對外連線
  • host only:VM自己存在於一個與世隔絕的虛擬網路中
詳細的解說 google 上可以找到一大把,先這樣吧~

這邊用的是NAT的方式,在Virtual Machine Settings 裡面 Network Adapter要選定 NAT

再來是叫出 Editor -> Virtual Network Editor ,預設情況下 VMnet8 的連接方式為 NAT 模式。
 點選底下的 Change Settings 按鈕設定 Subnet IP / Subnet Mask

設好這邊的 Subnet IP / Subnet Mask 就可以儲存離開了

在來到網路連線設定這邊選 VMnet8 -> 內容,按照下圖設定


接著回到 Linex 畫面,在 /etc/sysconfig/network-scripts/ifcfg-eth0 設定網路,這邊的 IP / NetMask / Gateway 得跟剛剛設定的對齊

改好後下「 service network restart 」重啟網路,用 ifconfig確認網卡設定是否正確,然後 ping 剛剛設定的IP看有沒有通,有的話就成功啦~ 可喜可賀~ 可喜可賀~

2018-06-08

maven settings.xml 樣板

Maven 設定檔 settings.xml 預設路徑會在 ${user.home}/.m2/repository 下
如果失蹤或是找不到的話可以用下面的樣板自己生一個出來


<?xml version="1.0" encoding="UTF-8"?>

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements.  See the NOTICE file
distributed with this work for additional information
regarding copyright ownership.  The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License.  You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied.  See the License for the
specific language governing permissions and limitations
under the License.
-->

<!--
 | This is the configuration file for Maven. It can be specified at two levels:
 |
 |  1. User Level. This settings.xml file provides configuration for a single
 |                 user, and is normally provided in
 |                 ${user.home}/.m2/settings.xml.
 |
 |                 NOTE: This location can be overridden with the CLI option:
 |
 |                 -s /path/to/user/settings.xml
 |
 |  2. Global Level. This settings.xml file provides configuration for all
 |                 Maven users on a machine (assuming they're all using the
 |                 same Maven installation). It's normally provided in
 |                 ${maven.home}/conf/settings.xml.
 |
 |                 NOTE: This location can be overridden with the CLI option:
 |
 |                 -gs /path/to/global/settings.xml
 |
 | The sections in this sample file are intended to give you a running start
 | at getting the most out of your Maven installation. Where appropriate, the
 | default values (values used when the setting is not specified) are provided.
 |
 |-->
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">

  <!-- localRepository
   | The path to the local repository maven will use to store artifacts.
   |
   | Default: ~/.m2/repository
  <localRepository>/path/to/local/repo</localRepository>
  -->


  <localRepository>C:\Users\user\.m2\repository</localRepository>

<build>
    <pluginManagement>
      <plugins>
        <plugin>
          <artifactId>maven-resources-plugin</artifactId>
          <version>2.6</version>
        </plugin>           
      </plugins>
    </pluginManagement>   
</build>

  <!-- interactiveMode
   | This will determine whether maven prompts you when it needs input. If set
   | to false, maven will use a sensible default value, perhaps based on some
   | other setting, for the parameter in question.
   |
   | Default: true
  <interactiveMode>true</interactiveMode>
  -->

  <!-- offline
   | Determines whether maven should attempt to connect to the network when
   | executing a build. This will have an effect on artifact downloads,
   | artifact deployment, and others.
   |
   | Default: false
  <offline>false</offline>
  -->

  <!-- pluginGroups
   | This is a list of additional group identifiers that will be searched when
   | resolving plugins by their prefix, i.e. when invoking a command line like
   | "mvn prefix:goal". Maven will automatically add the group identifiers
   | "org.apache.maven.plugins" and "org.codehaus.mojo" if these are not
   | already contained in the list.
   |-->
  <pluginGroups>
    <!-- pluginGroup
     | Specifies a further group identifier to use for plugin lookup.
    <pluginGroup>com.your.plugins</pluginGroup>
    -->
 
  </pluginGroups>

  <!-- proxies
   | This is a list of proxies which can be used on this machine to connect to
   | the network. Unless otherwise specified (by system property or command-
   | line switch), the first proxy specification in this list marked as active
   | will be used.
   |-->
  <proxies>
    <!-- proxy
     | Specification for one proxy, to be used in connecting to the network.
     |
    <proxy>
      <id>optional</id>
      <active>true</active>
      <protocol>http</protocol>
      <username>proxyuser</username>
      <password>proxypass</password>
      <host>proxy.host.net</host>
      <port>80</port>
      <nonProxyHosts>local.net|some.host.com</nonProxyHosts>
    </proxy>
    -->
  </proxies>

  <!-- servers
   | This is a list of authentication profiles, keyed by the server-id used
   | within the system. Authentication profiles can be used whenever maven must
   | make a connection to a remote server.
   |-->
  <servers>
    <!-- server
     | Specifies the authentication information to use when connecting to a
     | particular server, identified by a unique name within the system
     | (referred to by the 'id' attribute below).
     |
     | NOTE: You should either specify username/password OR
     |       privateKey/passphrase, since these pairings are used together.
     |
    <server>
      <id>deploymentRepo</id>
      <username>repouser</username>
      <password>repopwd</password>
    </server>
    -->

    <!-- Another sample, using keys to authenticate.
    <server>
      <id>siteServer</id>
      <privateKey>/path/to/private/key</privateKey>
      <passphrase>optional; leave empty if not used.</passphrase>
    </server>
    -->
  </servers>

  <!-- mirrors
   | This is a list of mirrors to be used in downloading artifacts from remote
   | repositories.
   |
   | It works like this: a POM may declare a repository to use in resolving
   | certain artifacts. However, this repository may have problems with heavy
   | traffic at times, so people have mirrored it to several places.
   |
   | That repository definition will have a unique id, so we can create a
   | mirror reference for that repository, to be used as an alternate download
   | site. The mirror site will be the preferred server for that repository.
   |-->
  <!--<mirrors>
     mirror
     | Specifies a repository mirror site to use instead of a given repository.
     | The repository that this mirror serves has an ID that matches the
     | mirrorOf element of this mirror. IDs are used for inheritance and direct
     | lookup purposes, and must be unique across the set of mirrors.
     |
    <mirror>
      <id>mirrorId</id>
      <mirrorOf>repositoryId</mirrorOf>
      <name>Human Readable Name for this Mirror.</name>
      <url>http://my.repository.com/repo/path</url>
    </mirror>
     -->

   <!-- <mirror>
        <id>nexus-aliyun</id>
        <mirrorOf>central</mirrorOf>
        <name>Nexus aliyun</name>
        <url>http://maven.aliyun.com/nexus/content/groups/public</url>
    </mirror>
  </mirrors>-->



  <!-- profiles
   | This is a list of profiles which can be activated in a variety of ways,
   | and which can modify the build process. Profiles provided in the
   | settings.xml are intended to provide local machine-specific paths and
   | repository locations which allow the build to work in the local
   | environment.
   |
   | For example, if you have an integration testing plugin - like cactus -
   | that needs to know where your Tomcat instance is installed, you can
   | provide a variable here such that the variable is dereferenced during the
   | build process to configure the cactus plugin.
   |
   | As noted above, profiles can be activated in a variety of ways. One
   | way - the activeProfiles section of this document (settings.xml) - will be
   | discussed later. Another way essentially relies on the detection of a
   | system property, either matching a particular value for the property, or
   | merely testing its existence. Profiles can also be activated by JDK
   | version prefix, where a value of '1.4' might activate a profile when the
   | build is executed on a JDK version of '1.4.2_07'. Finally, the list of
   | active profiles can be specified directly from the command line.
   |
   | NOTE: For profiles defined in the settings.xml, you are restricted to
   |       specifying only artifact repositories, plugin repositories, and
   |       free-form properties to be used as configuration variables for
   |       plugins in the POM.
   |
   |-->

  <profiles>
    <!-- profile
     | Specifies a set of introductions to the build process, to be activated
     | using one or more of the mechanisms described above. For inheritance
     | purposes, and to activate profiles via <activatedProfiles/> or the
     | command line, profiles have to have an ID that is unique.
     |
     | An encouraged best practice for profile identification is to use a
     | consistent naming convention for profiles, such as 'env-dev',
     | 'env-test', 'env-production', 'user-jdcasey', 'user-brett', etc. This
     | will make it more intuitive to understand what the set of introduced
     | profiles is attempting to accomplish, particularly when you only have a
     | list of profile id's for debug.
     |
     | This profile example uses the JDK version to trigger activation, and
     | provides a JDK-specific repo.
    <profile>
      <id>jdk-1.4</id>

      <activation>
        <jdk>1.4</jdk>
      </activation>

      <repositories>
        <repository>
          <id>jdk14</id>
          <name>Repository for JDK 1.4 builds</name>
          <url>http://www.myhost.com/maven/jdk14</url>
          <layout>default</layout>
          <snapshotPolicy>always</snapshotPolicy>
        </repository>
      </repositories>
    </profile>
    -->

    <!--
     | Here is another profile, activated by the system property 'target-env'
     | with a value of 'dev', which provides a specific path to the Tomcat
     | instance. To use this, your plugin configuration might hypothetically
     | look like:
     |
     | ...
     | <plugin>
     |   <groupId>org.myco.myplugins</groupId>
     |   <artifactId>myplugin</artifactId>
     |
     |   <configuration>
     |     <tomcatLocation>${tomcatPath}</tomcatLocation>
     |   </configuration>
     | </plugin>
     | ...
     |
     | NOTE: If you just wanted to inject this configuration whenever someone
     |       set 'target-env' to anything, you could just leave off the
     |       <value/> inside the activation-property.
     |
    <profile>
      <id>env-dev</id>

      <activation>
        <property>
          <name>target-env</name>
          <value>dev</value>
        </property>
      </activation>

      <properties>
        <tomcatPath>/path/to/tomcat/instance</tomcatPath>
      </properties>
    </profile>
    -->
  </profiles>

  <!-- activeProfiles
   | List of profiles that are active for all builds.
   |
  <activeProfiles>
    <activeProfile>alwaysActiveProfile</activeProfile>
    <activeProfile>anotherAlwaysActiveProfile</activeProfile>
  </activeProfiles>
  -->
</settings>

2017-02-10

Ubuntu 10.04 source.list 站點

deb http://old-releases.ubuntu.com/ubuntu/ jaunty main restricted universe multiversedeb http://old-releases.ubuntu.com/ubuntu/ jaunty-updates main restricted universe multiversedeb http://old-releases.ubuntu.com/ubuntu/ jaunty-security main restricted universe multiverse
deb-src http://old-releases.ubuntu.com/ubuntu/ jaunty main restricted universe multiversedeb-src http://old-releases.ubuntu.com/ubuntu/ jaunty-updates main restricted universe multiversedeb-src http://old-releases.ubuntu.com/ubuntu/ jaunty-security main restricted universe multiverse

ubuntu GPG error: NO_PUBKEY 解决方法

問題描述
做 apt-get update 時出現
W: GPG error: http://ppa.launchpad.net lucid Release: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY C2518248EEA14886

解決方法
  1. 連到 OpenPGP 公鑰伺服器,在 Search String 欄入 0xC2518248EEA14886 (這裡要修改成錯誤訊息裡的資訊,因為是 PUB 是十六進位,所以前面要加上 0x),在按 Search
  2. 在搜索結果裡點選 pub 欄的連結 ,得到公鑰內容
  3. 複製公鑰內容 (從 ----- 開始到 ----- 結束) 到本地文件中 (假設本地文件名稱為 key.txt)
  4. 執行命令 sudo apt-key add key.txt,新增公鑰,得到 OK 回覆即可
備註
如果重新 apt-get update 後遇到
The following signatures were invalid: BADSIG
問題,可以按下面步驟處理
  1. sudo apt-get clean
  2. cd /var/lib/apt
  3. sudo mv lists lists.old
  4. sudo mkdir -p lists/partial
  5. sudo apt-get clean
  6. sudo apt-get update
  7. 沒問題後,sudo rm -fr lists.old

Ref:
https://www.deadend.me/2016/10/12/fix-ubuntu-gpg-error-no-pubkey-problem/
https://blog.longwin.com.tw/2015/10/linux-fix-gpg-error-badsig-2015/

2016-12-16

[Ubuntu] bad ssh2 cipher spec error

要使用 SSH 連線到主機時發現碰到不能連線的情形
「ssh: connect to host 10.144.xxx.xxx port 22: Connection refused」

排除網路設定與防火牆問題後,從 /ver/log/syslog 發現 SSH 根本啟動失敗,錯誤訊息如下:
Dec 16 13:18:25 localhost init: ssh main process (834) terminated with status 255
Dec 16 13:18:25 localhost init: ssh main process ended, respawning

使用 「/usr/sbin/sshd -Dd」看到「bad ssh2 cipher spec error」錯誤訊息

參考 這篇文章

修改 /etc/ssh/sshd_config,將原本的 
Ciphers chacha20-poly1305@openssh.com,aes256-gcm@openssh.com,aes128-gcm@openssh.com,aes256-ctr,aes192-ctr,aes128-ctr
修改為
Ciphers aes256-cbc

重啟 SSH (service ssh start)
檢查 SSH 狀態 (service ssh status) 
即可正常使用 SSH 連線

2016-05-19

Fortify 說明 - XML External Entity Injection

問題種類:XML External Entity Injection
問題發生原因:XML解析器不會阻止及限制外部實體解析,如此可導致解析器遭受XML外部實體攻擊。

弱點解決辦法:
增加下面紅字兩行屬性,應以安全方式設定 XML 解析器,讓其不允許將外部實體包含在傳入的 XML 文件中。
documentBuilderFactory = DocumentBuilderFactory.newInstance();
documentBuilderFactory.setFeature("http://xml.org/sax/features/external-general-entities", false); documentBuilderFactory.setFeature("http://xml.org/sax/features/external-parameter-entities", false);documentBuilder = documentBuilderFactory.newDocumentBuilder();
document = documentBuilder.parse(filepath);

2016-04-21

Fortify 說明 - Portability Flaw: Locale Dependent Comparison

問題種類:Portability Flaw: Locale Dependent Comparison
問題說明 : 字串處理或轉換大小寫時,需要指定語系

錯誤程式範例:
if(args[0].toUpperCase().equals("abcd")){
}

修正的程式範例:
if(args[0].toUpperCase(Locale.TAIWAN).equals("abcd")){
}

或
Locale.setDefault(Locale.TRADITIONAL_CHINESE);

2015-11-19

Ubuntu 14.04.1 x VNC Server x 灰畫面

使用 Ubuntu x VNC Server 的時候常常碰到連進去是灰色畫面然後箭頭是個 X ,
雖然還是可以用,但是沒有華麗的 UI 用起來心情就不是很美麗,
Ubuntu 14.04.1 x VNC Server x 灰畫面 的情形普通的解法解不掉,
剛好花了不少時間找到一個可以用的,所以得把他記錄下來!

Step 1 安裝

sudo apt-get install gnome-core gnome-session-fallbacksudo apt-get install vnc4server

Step 2 Config 

sudo gedit .vnc/xstartup
#!/bin/sh[ -x /etc/vnc/xstartup ] && exec /etc/vnc/xstartup[ -r $HOME/.Xresources ] && xrdb $HOME/.Xresourcesxsetroot -solid pinkvncconfig -iconic &x-window-manager &gnome-panel &gnome-settings-daemon &metacity &nautilus -n &gnome-terminal &

Step 3 啟動 & 停止

vncserver  -geometry 1600x960 -depth 16 -dpi 100 :1vncserver -kill :1