Boost Your Download Speed with lftp Segmentation

Looking for a faster way to download files via sftp to a Linux machine? Try using "lftp" instead. This tool offers segmented downloading, which allows you to download a file using multiple connections.

To get started, install lftp with the following command:

yum -y install lftp

Then, use the following command to connect to your sftp server and download a file with 20 segments:

bash
lftp sftp://username@server_address
>pget -n 20 /path/to/file/bigfile.img

In our tests, we were able to download a 300 MB file in just 4 seconds with a good internet connection. By comparison, the same file took around 8 minutes with a single threaded connection.

To download an entire directory, use the "mirror" command with the "--use-pget-n" flag:

css
mirror --use-pget-n=20 /path/to/directory

By using 20 segments, you can speed up the downloading process and get your files faster.

Troubleshooting HDP-3.1 + Ambari-2.7.5 Cluster 'Add/Install Service' Failure: Tips and Solutions

Error During Cluster Add Services Operation? Try This Solution

If you encounter the following error message during a cluster add services operation:

resource_management.core.exceptions.ExecutionFailed: Execution of 'usermod -G ranger,hadoop,hadoop,ranger -g hadoop ranger' returned 6. usermod: user 'ranger' does not exist in /etc/passwd Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-1328.json', '/var/lib/ambari-agent/cache/stack-hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-1328.json', 'INFO', '/var/lib/ambari-agent/tmp', 'PROTOCOL_TLSv1_2', '']

There's a solution you can try:

1. Check that the current values are set to "true" by running the following command:

/var/lib/ambari-server/resources/scripts/configs.py -l c149-node1.coelab.cloudera.com --protocol http -t 8080 -u admin -p 'rraman' -a get -n c149 -c cluster-env | egrep 'ignore_groupsusers_create|fetch_nonlocal_groups'

2. If the values are not set to "true," you can set them by running the following commands:

/var/lib/ambari-server/resources/scripts/configs.py -l c149-node1.coelab.cloudera.com --protocol http -t 8080 -u admin -p 'rraman' -a set -n c149 -c cluster-env -k fetch_nonlocal_groups -v true /var/lib/ambari-server/resources/scripts/configs.py -l c149-node1.coelab.cloudera.com --protocol http -t 8080 -u admin -p 'rraman' -a set -n c149 -c cluster-env -k ignore_groupsusers_create -v true

3. Finally, repeat step 1 to check that the values have been updated.

How to Convert MyISAM Tables to InnoDB in MySQL: A Step-by-Step Query Guide

Convert MyISAM to InnoDB in MySQL with this query from information_schema.tables:

Use this MySQL query to prepare the script:

SELECT CONCAT('ALTER TABLE ', table_name, ' ENGINE=InnoDB;') FROM information_schema.tables WHERE table_schema = 'ranger' AND ENGINE = 'MyISAM';

The output will contain a list of ALTER TABLE statements that you need to run in the MySQL terminal to change your MyISAM tables to InnoDB. Here's an example output:

ALTER TABLE vx_trx_log ENGINE=InnoDB; ALTER TABLE x_access_type_def ENGINE=InnoDB; ALTER TABLE x_access_type_def_grants ENGINE=InnoDB; ... ALTER TABLE xa_access_audit ENGINE=InnoDB;

Follow these steps to convert your MyISAM tables to InnoDB and improve the performance and reliability of your MySQL database.

Step-by-Step Guide to Install MySQL 5.7 on RHEL7/CentOS7

Learn how to install MySQL 5.7.20 on RHEL7 / CentOS7 with this easy-to-follow guide:

Step 1: Download the MySQL installation bundle with the following command:

wget https://downloads.mysql.com/archives/get/p/23/file/mysql-5.7.20-1.el7.x86_64.rpm-bundle.tar

Step 2: Extract the bundle using the command:

tar -xvf mysql-5.7.20-1.el7.x86_64.rpm-bundle.tar

Step 3: Install MySQL and its dependencies using the following command from the extracted directory:

yum -y install mysql-community-server-5.7.20-1.el7.x86_64.rpm mysql-community-common-5.7.20-1.el7.x86_64.rpm mysql-community-client-5.7.20-1.el7.x86_64.rpm mysql-community-libs-5.7.20-1.el7.x86_64.rpm

Step 4: Before proceeding, clean up the content of the /var/lib/mysql/* directory. If necessary, take a backup of the directory.

Next, initialize the MySQL database using the command:

mysqld --initialize --user=mysql

Start the MySQL service with the command:

service mysqld start

Step 5: Retrieve the temporary root password from the log using the command:

cat /var/log/mysqld.log |grep -i 'temporary password'

Step 6: Log in to MySQL as root using the temporary password and change the root password using the following commands:

mysql -u root -p Enter password: )eYlFY_xK0fX

alter user 'root'@'localhost' identified by 'newpassword';

Step 7: Exit MySQL using the command:

quit

That's it! Follow these steps and you'll have MySQL 5.7.20 up and running on your RHEL7 / CentOS7 system in no time.

A Step-by-Step Guide to Exporting and Importing Apache Ranger Policies Between Clusters

Meta Description: This blog post provides a detailed guide on how to export and import Apache Ranger policies from one cluster to another, along with a python script that simplifies the process. The post includes pre-requisites, policy export and import steps, and shell scripts for each step.

Keywords: Apache Ranger, policies, export, import, clusters, python script, pre-requisites, policy export, policy import, shell script, step-by-step guide

Introduction: This blog post provides a step-by-step guide to exporting and importing Apache Ranger policies between clusters. The guide includes a python script that simplifies the process and shell scripts for each step. The post also includes pre-requisites and policy export and import steps.

• Pre-requisites: The post mentions that having the same Ranger groups and users available on both the clusters is necessary, as some of the ranger imports may fail if a user or group doesn't exist.
• 
  
• Policy export steps: The post provides detailed steps on how to export policies, including a command to extract the first 25000 Hive policy names, a command to export each policy in a separate file, and instructions on how to check the output and move the files to the target cluster.
• 
  
• Policy import steps: The post provides detailed steps on how to prepare the exported policies for import on the target cluster, including a command to change the source cluster's ranger hive repository name to the target cluster's name. It also provides a shell script for the policy import step, including instructions on how to run the script on the target cluster and check the policies in the Ranger UI.
• 
  

Step1:- Prepare the exported policies ready for import on target cluster. As my source cluster ranger hive repository name is "c349_hive" and i need to change those to "c149_hive" which is my target cluster ranger hive repository name. shell command to convert this :- This will change all the copied files and replace the values. sed -i -e 's/c349_hive/c149_hive/g' *.json Step2:- Prepare the shell script for import( Just change the target cluster Ranger host and port accordingly and /service/c149_hive/policy in URL) for i in `cat policy_names.txt`; do echo "curl -u admin:admin -H \"Content-Type: application/json\" -X POST http://c149-node4:6080/service/public/v2/api/policy -d @policy_$i.json ">> curl_import_policy.sh; done Step3:- Run the shell script "curl_import_policy.sh" on the target cluster to import the policies. nohup ./curl_import_policy.sh & The import should take time depending on the number of policies. Once finished you can check in target Ranger UI to check the policies.

Conclusion:

Exporting and importing Apache Ranger policies between clusters can be a complex task, but this guide and the accompanying python script can help simplify the process. By following the pre-requisites and the detailed policy export and import steps, users can ensure that their policies are successfully transferred between clusters.

Complete Guide: How to Install MySQL 5.7.20 on Rhel6/Centos6 - Step by Step Tutorial

For RHEL7/centos7  Follow here

Learn how to install mysql-5.7.20 on Rhel6/Centos6 with this easy step-by-step guide:

Step 1: Download the installation bundle First, download the mysql-5.7.20-1.el6.x86_64.rpm-bundle.tar file by running the following command:

wget https://downloads.mysql.com/archives/get/p/23/file/mysql-5.7.20-1.el6.x86_64.rpm-bundle.tar

Step 2: Extract the files Use the following command to extract the files from the bundle:

tar -xvf mysql-5.7.20-1.el6.x86_64.rpm-bundle.tar

Step 3: Install mysql packages Now that you have the files, install the mysql packages by running the following command from the extracted directory:

yum -y install mysql-community-server-5.7.20-1.el6.x86_64.rpm mysql-community-common-5.7.20-1.el6.x86_64.rpm mysql-community-client-5.7.20-1.el6.x86_64.rpm mysql-community-libs-5.7.20-1.el6.x86_64.rpm mysql-community-libs-5.7.20-1.el6.x86_64.rpm

Step 4: Initialize and start mysql service Before starting the mysql service, remove any content in the /var/lib/mysql/* directory. Then, initialize the mysql service by running the following command:

mysqld --initialize --user=mysql

Start the mysql service by running the following command:

service mysqld start

Step 5: Retrieve the temporary root password Use the following command to retrieve the temporary root password from the mysql log:

cat /var/log/mysqld.log |grep -i 'temporary password'

Step 6: Change the root password Log in as root using the temporary password you retrieved in the previous step:

mysql -u root -p

Then, change the root password using the following command:

mysql> alter user 'root'@'localhost' identified by 'new_password';

Step 7: Quit Finally, exit the mysql prompt by typing quit.