Автоматическое создание виртуальных кластеров Apache Spark в облачной среде Openstack
https://doi.org/10.15514/ISPRAS-2014-26(4)-3
Аннотация
Об авторах
О. Д. БорисенкоРоссия
Д. Ю. Турдаков
Россия
С. Д. Кузнецов
Россия
Список литературы
1. Страница проекта Apache Hadoop - http://hadoop.apache.org/
2. Страница проекта Cloudera CDH Apache Hadoop - http://www.cloudera.com/content/cloudera/en/products-and-services/cdh.html
3. Страница проекта Infinispan - http://infinispan.org/
4. Страница проекта Basho Riak - http://basho.com/riak/
5. Страница проекта Apache Spark - http://spark.apache.org/
6. M. Chowdhury, M. Zaharia, I. Stoica. Performance and Scalability of Broadcast in Spark. 2010.
7. Gu, Lei, and Huan Li. Memory or Time: Performance Evaluation for Iterative Operation on Hadoop and Spark. High Performance Computing and Communications & 2013 IEEE International Conference on Embedded and Ubiquitous Computing (HPCC_EUC), 2013 IEEE 10th International Conference on. IEEE, 2013.
8. Страница проекта VMWare Serengeti - http://www.vmware.com/hadoop/serengeti
9. Страница проекта Cloudera Manager - http://www.cloudera.com/content/cloudera/en/products-and-services/cloudera-enterprise/cloudera-manager.html
10. Страница проекта Openstack Sahara, план очередности разработки - https://wiki.openstack.org/wiki/Sahara/Roadmap
11. Foley, Matt. High Availability HDFS. 28th IEEE Conference on Massive Data Storage, MSST. Vol. 12. 2012.
12. Hunt, Patrick, et al. ZooKeeper: Wait-free Coordination for Internet-scale Systems. USENIX Annual Technical Conference. Vol. 8. 2010.
13. Massie, Matthew, B. Chun, and D. Culler. The ganglia distributed monitoring system: design, implementation, and experience. Parallel Computing 30.7 (2004): 817-840.
14. Страница сервиса Amazon Elastic Compute Cloud (EC2) - http://aws.amazon.com/ec2/
15. Creeger, Mache. Cloud Computing: An Overview. ACM Queue 7.5 2009.
16. Страница проекта Openstack Heat - https://wiki.openstack.org/wiki/Heat
17. Yokoyama, Shigetoshi, and Nobukazu Yoshioka. Cluster as a Service for self-deployable cloud applications. Cluster, Cloud and Grid Computing (CCGrid), 2012 12th IEEE/ACM International Symposium on. IEEE, 2012.
18. Страница проекта Chef http://www.getchef.com/
19. Страница проекта Salt http://www.saltstack.com/
20. Страница проекта Ansible http://www.ansible.com/home
21. Ожидает публикации. К. Чихрадзе, А. Коршунов, Н. Бузун, Н. Кузюрин. Использование модели социальной сети с сообществами пользователей для распределённой генерации случайных социальных графов. 10-я Международная конференция «Интеллектуализация обработки информации» 2014.
22. Apache Hadoop project web page - http://hadoop.apache.org/
23. Cloudera CDH Apache Hadoop project web page - http://www.cloudera.com/content/cloudera/en/products-and-services/cdh.html
24. Infinispan project web page - http://infinispan.org/
25. Basho Riak project web page - http://basho.com/riak/
26. Apache Spark project web page - http://spark.apache.org/
27. M. Chowdhury, M. Zaharia, I. Stoica. Performance and Scalability of Broadcast in Spark. 2010.
28. Gu, Lei, and Huan Li. Memory or Time: Performance Evaluation for Iterative Operation on Hadoop and Spark. High Performance Computing and Communications & 2013 IEEE International Conference on Embedded and Ubiquitous Computing (HPCC_EUC), 2013 IEEE 10th International Conference on. IEEE, 2013.
29. VMWare Serengeti project web page - http://www.vmware.com/hadoop/serengeti
30. Cloudera Manager project web page - http://www.cloudera.com/content/cloudera/en/products-and-services/cloudera-enterprise/cloudera-manager.html
31. Openstack Sahara project web page, roadmap - https://wiki.openstack.org/wiki/Sahara/Roadmap
32. Foley, Matt. High Availability HDFS. 28th IEEE Conference on Massive Data Storage, MSST. Vol. 12. 2012.
33. Hunt, Patrick, et al. ZooKeeper: Wait-free Coordination for Internet-scale Systems. USENIX Annual Technical Conference. Vol. 8. 2010.
34. Massie, Matthew, B. Chun, and D. Culler. The ganglia distributed monitoring system: design, implementation, and experience. Parallel Computing 30.7 (2004): 817-840.
35. Amazon Elastic Compute Cloud (EC2) service webpage - http://aws.amazon.com/ec2/
36. Creeger, Mache. Cloud Computing: An Overview. ACM Queue 7.5 2009.
37. Openstack Heat project web page - https://wiki.openstack.org/wiki/Heat
38. Yokoyama, Shigetoshi, and Nobukazu Yoshioka. Cluster as a Service for self-deployable cloud applications. Cluster, Cloud and Grid Computing (CCGrid), 2012 12th IEEE/ACM International Symposium on. IEEE, 2012.
39. Chef project web page - http://www.getchef.com/
40. Salt project web page - http://www.saltstack.com/
41. Ansible project web page - http://www.ansible.com/home
42. In print. K. Chikhradze, А. Korshunov, N. Buzun, N. Kuzyurin. Ispol'zovanie modeli sotsial'noj seti s soobshhestvami pol'zovatelej dlya raspredelyonnoj generatsii sluchajnykh sotsial'nykh grafov [On a model of social network with user communities for distributed generation of random social graphs]. 10-ya Mezhdunarodnaya konferentsiya «Intellektualizatsiya obrabotki informatsii» [10th International conference “Intelligent Information Processing”] 2014.
Рецензия
Для цитирования:
Борисенко О.Д., Турдаков Д.Ю., Кузнецов С.Д. Автоматическое создание виртуальных кластеров Apache Spark в облачной среде Openstack. Труды Института системного программирования РАН. 2014;26(4):33-44. https://doi.org/10.15514/ISPRAS-2014-26(4)-3
For citation:
Borisenko O., Turdakov D., Kuznetsov S. Automating cluster creation and management for Apache Spark in Openstack cloud. Proceedings of the Institute for System Programming of the RAS (Proceedings of ISP RAS). 2014;26(4):33-44. (In Russ.) https://doi.org/10.15514/ISPRAS-2014-26(4)-3