By Anil Vaidya
Big Data Juggernaut IV
The big data is taking big leaps with Spark based products. The cloud as well as on-premises solutions deploy spark in their offerings. I wrote earlier that Spark is taking over Hadoop as big data mainstay. The Spark needs to be supported by additional access mechanisms and the programming languages. Not surprisingly Python is rising to the occasion. A special version of python named ‘PySpark’ does this very well. PySpark is the programming interplay with provision for accessing Spark based datasets. It has built-in libraries that allows programmers to do computation of data stored under Apache Spark.
This simply means if someone is using Spark, he/she needs to work with PySpark too. Going further the PySpark being based on Python one needs to know bit of Python too. One of the easier ways to start working on PySpark is the use of Jupyter Notebook. By now you have gauged the number of different technologies have to integrate to be able to get into Big data project. It is imperative that one has to have a combination of Business mindset and liking for technological innovations.
Technology is developing at a rapid pace, beyond imagination. Number of people and companies working in this arena has been phenomenally high, also spread geographically all over the world. We, to be successful, have to keep an eye on these developments but also upgrade ourselves all the time. Just think of how many different technologies I brought together in this short blog, starting from Spark to Python, to PySpark and Jupyter Notebook, all within the ambit of BIG DATA.
Tags
- Uncategorized (5)
- Abbasali Gabula (1)
- Aditi Divatia (2)
- Anil Kulkarni (1)
- Anil Vaidya (14)
- Ashish Kumar Jha (1)
- Ashita Aggarwal (2)
- Atul Sethi (1)
- Balanced Leadership (1)
- Bindu Kulkarni (4)
- Deepa Krishnan (6)
- Equality (1)
- Harsh Mohan (1)
- Hemant Manuj (1)
- Jagdish Rattanani (2)
- Lata Dhir (1)
- M.Suresh Rao (4)
- Mita Dixit (3)
- Nirja Mattoo (1)
- Pallavi Mody (6)
- Preobroto Ganguly (1)
- R Jayaraman (38)
- R K Pattnaik (2)
- R. Jayaraman (1)
- R.Gopalakrishnan (2)
- Rajiv Agarwal (1)
- Rakhi Thakur (1)
- Ranjan Banerjee (2)
- Ratika Gore (2)
- Renuka Kamath (3)
- Rukaiya Joshi (1)
- Sajeev George (1)
- Sapna Malya (1)
- Sarabjeet Natesan (12)
- Sheila Roy (2)
- Snehal Shah (1)
- Sumita Datta (1)
- Suresh G. Lalwani (1)
- Surya Tahora (3)
- Sushmita Srivastava (1)
- Tulsi Jayakumar (4)
- Vanita Bhoola (2)
- Vasant Sivaraman (1)
- Vijay Sampath (3)
- Wisdom (1)
- Women's Day (1)
Most Viewed
Blog Archive
- June 2020 (1)
- May 2020 (1)
- October 2019 (1)
- August 2019 (1)
- May 2019 (1)
- February 2019 (2)
- December 2018 (1)
- November 2018 (1)
- September 2018 (4)
- August 2018 (3)
- July 2018 (1)
- June 2018 (4)
- May 2018 (2)
- March 2018 (3)
- December 2017 (6)
- September 2017 (6)
- August 2017 (1)
- July 2017 (4)
- May 2017 (4)
- April 2017 (4)
- March 2017 (5)
- January 2017 (3)
- December 2016 (3)
- November 2016 (9)
- October 2016 (17)
- September 2016 (48)
- August 2016 (7)
Comments
Big Data Analytics
Big Data is here to stay.
Add new comment