Integrating Task and Data Allocation Using a Parallel File System

• Hélio Crestana Guardia UFSCar
• Liria Matsumoto Sato USP

DOI: https://doi.org/10.5753/sbac-pad.2000.41232

Resumo

Parallel files combine several disks to provide increased storage capacity and fast data transfer between the primary and secondary memories. The use of parallel files by distributed applications however introduces new challenges to the parallel file system, which may inhibit the scalability of such applications. This work presents a study on the creation of a parallel file system that facilitates the development of distributed parallel applications with large amount of I/O operations. The proposed system was implemented and generated new tools that integrate file system operation with a mechanism for process creation and distribution. The evaluation of the system shows the viability of its utilization allowing the scalability of a distributed parallel application.

Palavras-chave: parallel I/O, distributed systems

Referências

ARPACI-DUSSEAU, Remzi H.; et al. Cluster I/O with River: Making the fast case common. In Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems, pages 10–22, Atlanta, GA, May 1999. ACM Press.

AVALANI, Bhavan; CHOUDHARY, Alok; FOSTER, Ian; KIRSHNAIYER, Rakesh. Integrating task and data parallelism using parallel I/O techniques. In Proceedings of the International Workshop on Parallel Processing, Bangalore, India, December 1994.

CORBETT, Peter F. et al. Proposal for a common parallel file system programming interface. [link], September 1996. Version 1.0.

CROCKETT, Thomas W. File Concepts for Parallel I/O. In Proceedings of Supercomputing ’89, p.574–579, 1989.

GUARDIA, Hélio C.; SATO, Lina M. NPFS: Um Sistema de Arquivos Paralelos em Rede. In Proceedings of the 17th Brazilian Symposium on Computer Network (SBRC), 1999.

GUARDIA, Hélio C. Considerações Sobre as Estratégias de um Sistema de Arquivos Paralelos Integrado ao Processamento Distribuído. PhD Thesis, EPUSP, 1999.

HUBER, Jay et al. PPFS: A high performance portable parallel file system. Technical Report UIUCDCS-R-95-1903, University of Illinois at Urbana-Champaign, January 1995.

KATZ, Randy H.; GIBSON, Garth A.; PATTERSON, David A. Disk System Architectures for High Performance Computing. In Proceedings of the IEEE, v.77, n.12, p.1842–1858, December 1989.

KOTZ, David. Disk-directed I/O for MIMD multiprocessors. Technical Report PCS-TR94-226, Dept. of Computer Science, Dartmouth College, July 1994. Revised November 8, 1994.

MACHE, Jens; et al. The impact of spatial layout of jobs on parallel I/O performance. In Proceedings of the Sixth Workshop on Input/Output in Parallel and Distributed Systems, pages 45–56, Atlanta, GA, May 1999. ACM Press.

MADHYASTHA, Tara M.; ELFORD, Christopher L.; REED, Daniel A. Optimizing input/output using adaptive file system policies. In Proceedings of the Fifth NASA Goddard Conference on Mass Storage Systems, pages II:493–514, September 1996.

OLDFIELD, Ron; KOTZ, David. Applications of parallel I/O. Technical Report PCS-TR98-337, Dept. of Computer Science, Dartmouth College, August 1998.

ROSTI, Emilia et al. The impact of I/O on program behavior and parallel scheduling. In Proceedings of the Joint International Conference on Measurement and Modeling of Computer Systems, pages 56–65. ACM Press, June 1998.

SIMITCI, Huseyin; REED, Daniel A. Adaptive disk striping for parallel input/output. In Proceedings of the Seventh NASA Goddard Conference on Mass Storage Systems, San Diego, CA, March 1999. IEEE Computer Society Press. To appear.

STOCKINGER, H. Classification of Parallel Input/Output Products. In Proceedings of the PDPTA ’98, 1998.