æ¡çšåãããŒã¿ã»åæè©äŸ¡ãã¹ã
åæãçµ±èšãããŒã¿æºåãå¯èŠåãããã³ã¢ããªã¹ãã䜿çšãããã©ãããã©ãŒã ã察象ãšããäžçŽã®ããŒã¿ã»åæãã¹ã22çš®é¡ãæ¯èŒã§ããŸãã
ç°¡åã«èšããš
ãã®22çš®é¡ã®éžæåŒãã¹ãã¯äžçŽã¬ãã«ã§ãçµ±èšãããŒã¿æºåãåæãå¯èŠåãããŒã¿ãã©ãããã©ãŒã ã®ã¹ãã«ãè©äŸ¡ããŸããã¢ããªã¹ããããŒã¿ãµã€ãšã³ãã£ã¹ãããšã³ãžãã¢ãBIã»ã¬ããŒãéçºè ãªã©ãããŒã¿ãæ±ãè·çš®ã®æ¡çšã«é©ããŠããŸããè·åå 容ã«åã£ããã¹ããéžã³ãçµæã¯äººã確èªããããã®åèæ å ±ãšããŠæ±ã£ãŠãã ããã
22ä»¶ã®ãã¹ã
Amazon Web Services (AWS)ã§ã®ããŒã¿åæ
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠãããã°ã©ãã³ã°ã¹ãã«Azureã§ã®ããŒã¿åæ
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠããœãããŠã§ã¢ã¹ãã«Excelã§ã®ããŒã¿åæ
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠããœãããŠã§ã¢ã¹ãã«Google Cloud Platform (GCP)ã§ã®ããŒã¿åæ
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠãããã°ã©ãã³ã°ã¹ãã«Matplotlib
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠãããã°ã©ãã³ã°ã¹ãã«NumPy
10å · 16å · äžçŽ
éžæåŒ
ãã¹ããèŠãããã°ã©ãã³ã°ã¹ãã«Pandas
12å · 18å · äžçŽ
éžæåŒ
ãã¹ããèŠããœãããŠã§ã¢ã¹ãã«Power BI
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠãããã°ã©ãã³ã°ã¹ãã«PySpark
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠãããã°ã©ãã³ã°ã¹ãã«Rã䜿ã£ãããŒã¿åæ
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠããœãããŠã§ã¢ã¹ãã«Snowflake
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠããœãããŠã§ã¢ã¹ãã«Tableau
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠãããã°ã©ãã³ã°ã¹ãã«ããŒã¿ãšã³ãžãã¢åãApache Spark
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠãè·çš®å¥ã¹ãã«ããŒã¿ãµã€ãšã³ã¹
10å · 20å · äžçŽ
éžæåŒ
ãã¹ããèŠãè·çš®å¥ã¹ãã«ããŒã¿ã¹ããŒãªãŒããªã³ã°
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠãè·çš®å¥ã¹ãã«ããŒã¿ã«åºã¥ãæææ±ºå®
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠãè·çš®å¥ã¹ãã«ããŒã¿ã®åãæ±ã
10å · 12å · äžçŽ
éžæåŒ
ãã¹ããèŠãè·çš®å¥ã¹ãã«ããŒã¿ã¢ããªã³ã°ã®æŠå¿µ
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠãè·çš®å¥ã¹ãã«ããŒã¿ã©ã³ã°ãªã³ã°
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠãè·çš®å¥ã¹ãã«ããŒã¿æœåºã»å€æã»èªã¿èŸŒã¿ïŒETLïŒ
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠãè·çš®å¥ã¹ãã«æ¢çŽ¢çããŒã¿åæ
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠãè·çš®å¥ã¹ãã«çµ±èšã»ç¢ºçã®åºç€
10å · 15å · äžçŽ
éžæåŒ
ãã¹ããèŠãã«ããŽãªãŒå¥ãã¹ã
- è·çš®å¥ã¹ãã« (9ä»¶ã®ãã¹ã)
- ããã°ã©ãã³ã°ã¹ãã« (7ä»¶ã®ãã¹ã)
- ãœãããŠã§ã¢ã¹ãã« (6ä»¶ã®ãã¹ã)
ããŒã¿ã»åæã®ãã¹ãã§è©äŸ¡ããå 容
22çš®é¡ã®ãã¹ãã§ã¯ãããŒã¿æ¥åå šè¬ã«ãããäžçŽã¹ãã«ãè©äŸ¡ããŸããçµ±èšçãªèãæ¹ããã¿ãŒã³ã®è§£éã«ã¯ãçµ±èšåŠãšç¢ºçè«ã®åºç€ãŸãã¯æ¢çŽ¢çããŒã¿åæãæ€èšããŠãã ãããããŒã¿ãµã€ãšã³ã¹ã§ã¯ãçµ±èšçãªèãæ¹ãã³ãŒãã䜿ã£ãããŒã¿å å·¥ãæ©æ¢°åŠç¿ã¢ãã«ã®æ§ç¯ããã¥ãŒã©ã«ãããã¯ãŒã¯ã®ãã¬ãŒãã³ã°ãæ±ããŸãã
ããŒã¿ã»ããã®æºåã«ã¯ãããŒã¿ã®è§£æãã¯ãªãŒãã³ã°ã圢ç¶å€æŽãçµåãæ€èšŒãè©äŸ¡ããããŒã¿å å·¥ãé©ããŠããŸããããŒã¿æœåºã»å€æã»èªã¿èŸŒã¿ïŒETLïŒã¯ãããŒã¿ã®èªã¿èŸŒã¿èšèšãšå®è¡ãåºåã®ç¢ºèªããã€ãã©ã€ã³ã®ä¿¡é Œæ§ã«çŠç¹ãåœãŠãŸããããŒã¿ã¢ããªã³ã°ã®æŠå¿µã§ã¯ããªã¬ãŒã·ã§ã³ãããŒãæ£èŠåãæŠå¿µã»è«çã»ç©çã®åã¬ãã«ã®ã¢ãã«ãæ±ããŸãã
ã»ãã«ããç¹å®ã®èšèªãã©ã€ãã©ãªããã©ãããã©ãŒã ãæ±ããã¹ãããããŸããPythonã®ã©ã€ãã©ãªã«ã¯åå¥ã®ãã¹ãããããé åãæ±ãNumPyããã°ã©ãäœæãæ±ãMatplotlibãªã©ããããŸããSnowflakeã§ã¯ãããŒã¿ãŠã§ã¢ããŠã¹ã®èšèšãSQLãããŒã¿ã®èªã¿èŸŒã¿ãããã©ãŒãã³ã¹èª¿æŽãæ±ããŸããããŒã¿ã®è§£éãæææ±ºå®ã®äŒéã«çŠç¹ãåœãŠããã¹ãã«ã¯ãããŒã¿ã«åºã¥ãæææ±ºå®ãããŒã¿ã¹ããŒãªãŒããªã³ã°ããããŸããPandasãé€ãåãã¹ãã®æèŠæéã¯10åã§ããPandasã¯12åã§ãããã¹ãŠéžæåŒã§ãã
è·åå 容ã«åãããŠãã¹ããéžã¶
è·åã§æå®ãããŠããæ¥åãããŒã«ã«åãããŠãã¹ããéžã³ãŸããããSparkãã€ãã©ã€ã³ãæ§ç¯ã»ä¿å®ããããŒã¿ãšã³ãžãã¢ãã¢ããªãã£ã¯ã¹ãšã³ãžãã¢ã«ã¯ãããŒã¿ãšã³ãžãã¢åãApache Sparkãé©ããŠããŸããSparkãžã§ããDataFrame倿ããã¥ãŒãã³ã°ãã¹ããªãŒãã³ã°ãè©äŸ¡ããŸããPySparkã¯ãããŒã¿ãšã³ãžãã¢ãããŒã¿ãµã€ãšã³ãã£ã¹ããã¢ããªã¹ããªã©ãäžçŽã¬ãã«ã®PySparkã¹ãã«ãå¿ èŠãªè·åã«é©ããŠããŸãã衚圢åŒããŒã¿ãPythonã§æ±ãæ¥åã«ã¯ãããŒã¿ã®éžæãã¯ãªãŒãã³ã°ãçµåãèŠçŽãè¡ãã³ãŒãã®èªè§£ã»èšè¿°åãæž¬ãPandasãé©ããŠããŸãã
ç¹å®ã®ã¯ã©ãŠããã©ãããã©ãŒã äžã§ããŒã¿ãæºåã»åæããã¢ããªã¹ãã«ã¯ã察å¿ãããã¹ããéžã³ãŸããããAmazon Web Services (AWS)ã§ã®ããŒã¿åæãAzureã§ã®ããŒã¿åæãGoogle Cloud Platform (GCP)ã§ã®ããŒã¿åæããããŸããããããããã®ãã©ãããã©ãŒã ã®åæãµãŒãã¹ãäžçŽã¬ãã«ã§æ±ãæ¥åãæ³å®ããŠããŸãã
ã¬ããŒãäœæãæ ãè·åã«ã¯ããµãŒãã¹ãPower QueryãããŒã¿ã¢ãã«ãDAXãã¬ããŒãã®ããžã¥ã¢ã«ãè©äŸ¡ããPower BIãé©ããŠããŸããTableauã§ã¯ãããŒã¿ã®æºåãèšç®ãã°ã©ããã€ã³ã¿ã©ã¯ãã£ããªããã·ã¥ããŒããæ±ããŸããã¹ãã¬ããã·ãŒãã®ããŒã¿ãæ°å€ãã°ã©ãã«ãŸãšããè·åã«ã¯ãExcelã§ã®ããŒã¿åæãçŽæ¥åœ¹ç«ã¡ãŸãããããã®ãã¹ãã¯äžçŽã¬ãã«ã§ããè·åã§æ±ããããã¬ãã«ãšæ¥åã«åãå Žåã«éžã³ãŸãããã
è·åã«åãããŠãã¹ããçµã¿åããã
åãæ¥åãè€æ°ã®ãã¹ãã§è©äŸ¡ããã®ã§ã¯ãªããè·åã®ç°ãªãé åãã«ããŒãããã¹ããå°æ°éžã³ãŸããããããŒã¿ã»ãããæ¢çŽ¢ããŠçµæãçºè¡šããããŒã¿ã¢ããªã¹ãã®è·åã«ã¯ãæ¢çŽ¢çããŒã¿åæãšããŒã¿ã¹ããŒãªãŒããªã³ã°ãçµã¿åãããããŸããç¹å®ã®ã¬ããŒãäœæããŒã«ã䜿ãè·åã§ããã°ãé¢ä¿ã®ãªããã©ãããã©ãŒã ã®ãã¹ãã§ã¯ãªããPower BIãTableauãªã©è©²åœãããã¹ãã远å ããŸãããã
ãã€ãã©ã€ã³ãæ±ãããŒã¿ãšã³ãžãã¢ã«ã¯ãããŒã¿å å·¥ãšé¢é£ããETLãŸãã¯Sparkã®ãã¹ããçµã¿åãããŸããããããŒã¿æ§é ã®èšèšãã¬ãã¥ãŒãè·åã«å«ãŸããå Žåã¯ããããŒã¿ã¢ããªã³ã°ã®æŠå¿µãã远å ããŸããããŒã¿ã掻çšããäºæ¥äžã®å€æãæ ãè·åã«ã¯ãããŒã¿ã«åºã¥ãæææ±ºå®ãããã¢ãã«ãæ§ç¯ããè·åã«ã¯ãããŒã¿ãµã€ãšã³ã¹ããŸãã¯ãçµ±èšåŠãšç¢ºçè«ã®åºç€ããé©ããŠããå ŽåããããŸãã
ãããã¯ãã¹ãŠäžçŽã¬ãã«ã®ãã¹ãã§ãåçŽãšäžçŽãåºå¥ãããã®ã§ã¯ãããŸãããè·åã«å¿ èŠãªå ·äœçãªæ¥åãããŒã«ã«æ²¿ã£ãŠçµã¿åãããŸããããåçµæã¯ãæ¡çšããã»ã¹å šäœã®äžã§ãè©äŸ¡å¯Ÿè±¡ã®ã¹ãã«ã«é¢ããåèæ å ±ãšããŠç¢ºèªããŠãã ãããã¹ã³ã¢ã¯äººãæ€èšããããã®ã·ã°ãã«ã§ãããããã ãã§ååŠãæ¡çšã決ãããã®ã§ã¯ãããŸããã
èšåã®å ¬å¹³æ§
ãã¹ãã®å 容ã«å¿ããŠãããŸããŸãªå¯Ÿçãè¬ããŠããŸããå€ãã®ãã¹ãã§ã¯ã泚æãã»ã³ãµãŒã®æž¬å®å€ãåºè·ã蚪åãã¢ã³ã±ãŒãã®åçãªã©ãäžç«çãªäŸãããŸããŸãªèæ¯ã®äººåãšãšãã«äœ¿çšããŠããŸããå¿ èŠã®ãªãå Žåã¯ãé貚èšå·ãæ £çšå¥ãã¹ããŒããå°ååºæã®ç¥èãèšåã«å«ããŸãããå¹³æãªè¡šçŸã«ãããè±èªåãèªè§£é床ã§ã¯ãªããè©äŸ¡å¯Ÿè±¡ã®ã¹ãã«ã«çŠç¹ãåœãŠãããããŠããŸãã
ã³ãŒããæ±ããã¹ãã§ã¯çãã³ãŒãäŸã䜿ãããœãããŠã§ã¢ã®ããŒãžã§ã³ã補åã«ãã£ãŠæåãç°ãªãå 容ã¯ã§ããã ãé¿ããŠããŸããèšåã«ãã£ãŠã¯ãåæãšãªãèšå®ãæåãæèšããŠããŸãããRã«ããããŒã¿åæãã§ã¯åèšåã«å°ããªããŒã¿ã»ããã瀺ããGoogle Cloudã®èšåã«ã¯å¿ èŠãªè¡šãã¯ãšãªãçµ±èšæ å ±ãå«ããŠããŸããçµ±èšã«é¢ããèšåã§ã¯ãèšå·ã䜿ãå Žåã«ãã®æå³ã説æããå¿ èŠãªè¡šã®å€ãæç€ºããŸããã°ã©ããããžã¥ã¢ã«ã䜿ãèšåã§ã¯ãè²ã ãã«é Œãããã©ãã«ãæ°å€ãããŒã«ãŒããã¿ãŒã³ãç·ã®çš®é¡ã䜿ã£ãŠããŸãã
忀æã®å®çšé¢ã§ã®é æ ®ã¯ãã¹ãã«ãã£ãŠç°ãªããŸããETLãRãå«ãè€æ°ã®ãã¹ãã§ã¯ãå šåæ€è ãã¡ã¢çšçŽã䜿ããŸãããçµ±èšåŠãšç¢ºçè«ã®åºç€ããšãããŒã¿ã®åãæ±ããã§ã¯ãå šåæ€è ã«é»åãçšæãããŠããŸããExcelãPower BIãTableauã§ã¯ãã°ã©ãã衚ãååãªå€§ããã§èªãããããããœã³ã³ã®äœ¿çšãæ±ããŠããŸãããã¹ããéžã¶éã¯åå¥ã®è©³çްã確èªããããããã®çµæãæèã«æ²¿ã£ãŠè§£éããŠãã ããã
ãããã質å
ããŒã¿ã¢ããªã¹ãã®è·åã«ã¯ã©ã®ããŒã¿ã»åæãã¹ããé©ããŠããŸããïŒ+
è·åå 容ã«åãããŠéžã³ãŸãããããæ¢çŽ¢çããŒã¿åæãã§ã¯ãçµ±èšãã°ã©ãã䜿ã£ãããŒã¿ã®èªã¿åããšèŠçŽãè©äŸ¡ããŸãããããŒã¿å å·¥ãã¯ããŒã¿ã®æºåãæ±ãããããŒã¿ã¹ããŒãªãŒããªã³ã°ãã¯ã°ã©ããããžã¥ã¢ã«ãåæçµæãæææ±ºå®ã«çŠç¹ãåœãŠãŸããPower BIãTableauã«ããããã·ã¥ããŒãäœæããŸãã¯Rã§ã®åæãªã©ããã®ããŒã«ãè·åã§äœ¿ãå Žåã«éããããŒã«ã®ãã¹ãã远å ããŸãããã
ããŒã¿ã»åæã®æ¡çšãã¹ãã¯åçŽè åãã§ããããããšãäžçŽè åãã§ããïŒ+
22çš®é¡ã®ãã¹ãã¯ãã¹ãŠäžçŽã¬ãã«ã§ãåçŽçãäžçŽçã¯ãããŸãããããŒã¿å ¥åãããŒã¿ã¢ã·ã¹ã¿ã³ãã®è·çš®ã«ã¯ããããŒã¿ã®åãæ±ãããæãé©ããŠããŸããããŒã¿ãæ£ç¢ºã§äžè²«ããç¶æ ã«ä¿ã€æ¹æ³ãåºæ¬çãªåæãã°ã©ãã®èªã¿åããæ±ããŸãã
éçšäž»ã¯ããŒã¿ã»åæã®ãã¹ããã©ã®ããã«çµã¿åããããšããã§ããïŒ+
è·åã®ç°ãªã責任ç¯å²ãã«ããŒãããã¹ããéžã³ãŸããããããšãã°ãããŒã¿ãæ¢çŽ¢ããŠçµæãçºè¡šããã¢ããªã¹ãã«ã¯ããæ¢çŽ¢çããŒã¿åæããšãããŒã¿ã¹ããŒãªãŒããªã³ã°ããçµã¿åãããããšãã§ããŸãããã€ãã©ã€ã³æ¥åã«ã¯ãé¢é£ããETLãŸãã¯Sparkã®ãã¹ããšãããŒã¿å å·¥ããçµã¿åãããããŒã¿ã¢ããªã³ã°ãè·åã«å«ãŸããå Žåã¯ãããŒã¿ã¢ããªã³ã°ã®æŠå¿µãã远å ããŸãããã
ç¹å®ã®ãœãããŠã§ã¢ã®ç¥èãäžèŠãªããŒã¿ãã¹ãã¯ã©ãã§ããïŒ+
ãããŒã¿ã«åºã¥ãæææ±ºå®ãã§ã¯ç¹å®ã®ãœãããŠã§ã¢ã®ç¥èã¯å¿ èŠãªãããããŒã¿ã¹ããŒãªãŒããªã³ã°ãã§ãç¹å®ã®ãœãããŠã§ã¢ã¯æå®ãããŠããŸããããæ¢çŽ¢çããŒã¿åæãã«ã¯ã³ãŒããç»å ŽããããããŒã¿ã®åãæ±ããã§ãããŒã«ã補åã®ç¥èã¯äžèŠã§ãããããããã¹ãã¯äœ¿çšããŒã«ãè·åã«ãã£ãŠç°ãªãå Žåã«é©ããŠããŸããäžæ¹ããã©ãããã©ãŒã ã®ãã¹ãã¯ç¹å®ã®è£œåã䜿ãè·åã«é©ããŠããŸãã
ããŒã¿ã»åæã®ãã¹ãã®æèŠæéã¯ã©ã®ãããã§ããïŒ+
Pandasãé€ãåãã¹ãã®æèŠæéã¯10åã§ããPandasã¯12åã§ãããã¹ãŠéžæåŒã§ã3ã€çµã¿åãããå Žåã®æèŠæéã¯çŽ30ïœ32åã§ããPandasããRã«ããããŒã¿åæããå«ã5ã€ã®ãã¹ãã§ã¯ãã¹ããŒããã©ã³ã§ã¯ãªãããœã³ã³ãå¿ èŠã§ãã
ããŒã¿ã»åæã®ãã¹ãã§ã¯ãã©ã®ããã«å ¬å¹³æ§ã«é æ ®ããŠããŸããïŒ+
å ¬å¹³æ§ãžã®é æ ®ã¯ãã¹ãã«ãã£ãŠç°ãªããŸãããå€ãã¯äžç«çãªããŒã¿ãšå¹³æãªè¡šçŸã䜿ããäžèŠãªå°åã»æåã®ç¥èãæ±ããŸãããã°ã©ãã«ã¯èª¬æãã©ãã«ããããè²ã ãã§çããæ±ºãŸããªãããã«ããŠããŸããè€æ°ã®ãã¹ãã§ã¯å šåæ€è ãã¡ã¢çšçŽã䜿ããçµ±èšã®ãã¹ãã§ã¯ç»é¢äžã®é»åãå©çšã§ããŸãã
ãããã®ãã¹ãããã¢ã»ã¹ã¡ã³ããäœæ
è·çš®ã«åã£ããã¹ããéžã³ãåè£è ãæåŸ ããŠçµæãæ¯èŒããŸãããã