24
1 Importing data from MySQL to HDFS using Sqoop a) MySQL Installation on Windows 1. Download MySQL Installer and unzip it. http://dev.mysql.com/downloads/mysql/

Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

  • Upload
    others

  • View
    3

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

1

ImportingdatafromMySQLtoHDFSusingSqoop

a) MySQLInstallationonWindows1. DownloadMySQLInstallerandunzipit.

http://dev.mysql.com/downloads/mysql/

Page 2: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

2

2. DoubleclickontheextractedfileandclickonInstallMysqlProducts.

Page 3: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

3

3. ClickNext

Page 4: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

4

4. Clicknext.

Page 5: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

5

5. ClickExecute.

Page 6: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

6

6. ClickNext

Page 7: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

7

7. ClickExecute.

Page 8: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

8

8. ClickNext.

Page 9: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

9

9. ClickNext.

Page 10: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

10

10. ClickNext.

Page 11: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

11

11. Setpasswordforrootuserandclicknext.

Page 12: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

12

12. ClickNext.

Page 13: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

13

13. ClickNext.

Page 14: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

14

14. ClickNext.

Page 15: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

15

15. ClickNext.

Page 16: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

16

16. OpenMysqlCommandLineClient,givepasswordandmysqlcommandlinewillopen.Nowwecancreatetables,databasesetc.

Page 17: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

17

17. Createanewdatabaseandusenewdatabaseusingcommandgivenbelow.

createdatabaseDeZyre;

useDeZyre;

18. Createatableandinsertrecordsasgivenbelow.

CREATETABLEplayer(

player_idINTNOTNULLAUTO_INCREMENT,

player_nameVARCHAR(100)NOTNULL,

PRIMARYKEY(player_id)

);

insertintoplayer(player_name)values("Sachin"),("Dravid"),("Dhoni"),("Ganguly"),("Yuvraj"),("Nehra"),("Singh");

Page 18: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

18

Page 19: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

19

19. Use“ipconfig”commandtofindoutIPofmysqlserver.

Page 20: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

20

ImportingDatafromMySQLtoHDFS

20. FindoutIPoftheserverwhereyouwanttoruntheSqoop.GrantprivilegetothatserveronMySQLserver.Let’ssayIPoftheserverwhereSqoopwillberunningis'192.168.138.128'.TograntprivilegeasshownaboveopenMySQLclientandrunfollowingcommand.TogetIPofVMmachine,FirecommandIfconfig

grantallprivilegeson*.*to'root'@'192.168.2.164'IDENTIFIEDBY'password'WITHGRANTOPTION

Page 21: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

21

21. ForimportingdatafromSQLtoHDFSusebelowcommandonClouderaVM.

sqoopimport--connectjdbc:mysql://192.168.138.1/dezyre--tableplayer--usernameroot-P--target-dir/user/cloudera/sqoopOut1-m1

Page 22: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

22

22. Onceabovecommandiscompleteyoucanseedatain/user/cloudera/sqoopOut1directory.

ExportdatafromHDFStoMySQL

23. Runfollowingcommandforcreatinganewdirectoryandcreatingdata.

hadoopdfs-mkdir/user/cloudera/sqoopOut2/

viplayer.csv

hadoopdfs-putplayer.csv/user/cloudera/sqoopOut2/

hadoopdfs-cat/user/cloudera/sqoopOut2/player.csv

Page 23: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

23

24. Runfollowingcommandforexportingdata.

sqoopexport--connectjdbc:mysql://192.168.138.1/dezyre--tableplayer--usernameroot-P--export-dir/user/cloudera/sqoopOut2/-m1

Page 24: Importing data from MySQL to HDFS using Sqoop€¦ · 20.Find out IP of the server where you want to run the Sqoop. Grant privilege to that server on MySQL server. Let’s say IP

24

25. SeefromMySQLclientifdataexported.

ImportDatatoHive

sqoopimport--connectjdbc:mysql://192.168.2.1/dezyre--usernameroot--passwordpassword--tableplayer--hive-import