Event: Large Data in MATLAB: A Seismic Data Processing Case Study
Date: February 23, 2011
Time*: 10:00 am Eastern Savings Time (GMT -04:00, New York)
Date: February 23, 2011
Time*: 10:00 am Eastern Savings Time (GMT -04:00, New York)
This is to introduce how to deal with very large data set in Matlab.
Each sub-data file about 1GB, but totally it might be up to 20GB.
The lecture uses a customerized reader "SegYFileReader" to read the segy file
data used in the web-seminar: software.seg.org: 2D datasets: 2004 BP Velocity Benchmark
Some Q& A:
[Q] Is it possible to parallelize a large matrix multiplication in the same manner? I have 2 cores and I'd like to split the calculation into two parallel processes to improve speed.
[A]: There are two ways to do this.
1) Use distributed arrays, which will allocate the data across distributed memory and supports math operations such as matrix multiply, fft, etc.
2) Write your matrix multiply algorithm to operate in parallel, using parfor or spmd (parallel commands).
[Q] : Could you give more info about the wave eq solver? Which wave eq do you use (elastic, acoustic)? Order of differences? Grid sizes? Running times (on what kind of computer)? Is the code written in matlab or some "faster" alternative (c fortran)?
[A]: It's a 4th order finite difference solution to the elastic wave equation written in MATLAB. There is also a naive version written for the GPU (stay tuned)
[Q]Is writing from multiple workers to a memory-mapped file supported?
[A]Yes, you can do this but you may need to pay attention simultaneous disk writes. Your OS may handle this, but for large writes you may need to implement a queue for writes at the same time.
[Q]Is this presentation going to go online very soon. Can you also make your matlab codes available so I see how you make some of these parallel computing
[A]A recording of the webinar will be posted in 2-3 days. The code will be posted to MATALB Central after the webinar, and will appear in the file exchange in 3-5 days.
[Q] Do you have any advice for writing large .wav files to disk? I have been struggling with writing large audio files to disk because of memory limitations of wavwrite and would appreciate any insight on this.
[A]We have new support for audio files that support this. I used a video writer in the webinar that handles writting frames of video. There is also one for audio files. You may also be interested in system objects which provide more support here as well.
[Q]Are you planning to offer a webinar to read SEGY files?
[A]Currently no plans to offer one specifically on SEGY files. Our consulting services group often works with customers to solve SEGY file import questions (since there are many nonstandard formats out there)
[Q]do you have any documentation on this topic?
[A]You can find the format specification for SEGY standards on the web. Reading/writting from them is done using fread in MATLAB. There is extensive documentation on file i/o operaionts in MATLAB for text and binary file i/o
[Q]Do you use GeForce or TESLA GPU?
[A] We're using the TESLA GPU. To use GPU capabilities, you need to have a gpu that supports double precision, compute capability of 1.3. NVIDA TESLA series 10 and 20 are examples that work. Our website list more detials on supported GPUs.
[Q]Is GPU processing supported for Matlab Embedded for Simulink and RTW for xPC?
[A]There currently is no support for embedded applications to use the GPU.
[Q](re:wavwrite) is this new support in 7.12? I'm running 7.11 R2010b and I don't see anything new in the Help as far as wavwrite/wavread is concerned. Could you specify what you mean by "new support" and point me in the right direction?
[A]Look for system objects, it's supported in signal processing toolbox which is why you may not see it if you don't have this toolbox. Also look for MultimediaFileWrite which supports it as well.
[Q]Are the codes going to be posted under recent files (in matlab central)?
[A]Yes, they will appear there.
[Q]When you talked about 56 cores, did you mean multiple cores on multiple machines, or it is all about one machine?
[A]It was 56 cores spread across multiple machines, the max of which had 8 cores on each. 56 was the available number, we have a 64 core capability but some machines weren't available at the time.
[Q]Is a special toolbox required to use the gpu functionality?
[A]Parallel Computing Toolbox provides GPU functionality.
[Q]what are symptoms that the GPU is overloaded with too many threads or too much memory
[A] It would generally show up a slower processing speeds, but depends on your particular problem you're trying to solve on the GPU. Not all problems scale well with GPUs.
The demo files will be posted to MATLAB central (matlabcentral.com) after the webinar. It usually takes 3-5 days to show up on the file exchange. The title will be the same as this webinar.
PDF copies of the slides will be made available upon request after the webinar. Contact your account representative, or support@mathworks.com to request a copy.
[Q] Is it possible to parallelize a large matrix multiplication in the same manner? I have 2 cores and I'd like to split the calculation into two parallel processes to improve speed.
[A]: There are two ways to do this.
1) Use distributed arrays, which will allocate the data across distributed memory and supports math operations such as matrix multiply, fft, etc.
2) Write your matrix multiply algorithm to operate in parallel, using parfor or spmd (parallel commands).
[Q] : Could you give more info about the wave eq solver? Which wave eq do you use (elastic, acoustic)? Order of differences? Grid sizes? Running times (on what kind of computer)? Is the code written in matlab or some "faster" alternative (c fortran)?
[A]: It's a 4th order finite difference solution to the elastic wave equation written in MATLAB. There is also a naive version written for the GPU (stay tuned)
[Q]Is writing from multiple workers to a memory-mapped file supported?
[A]Yes, you can do this but you may need to pay attention simultaneous disk writes. Your OS may handle this, but for large writes you may need to implement a queue for writes at the same time.
[Q]Is this presentation going to go online very soon. Can you also make your matlab codes available so I see how you make some of these parallel computing
[A]A recording of the webinar will be posted in 2-3 days. The code will be posted to MATALB Central after the webinar, and will appear in the file exchange in 3-5 days.
[Q] Do you have any advice for writing large .wav files to disk? I have been struggling with writing large audio files to disk because of memory limitations of wavwrite and would appreciate any insight on this.
[A]We have new support for audio files that support this. I used a video writer in the webinar that handles writting frames of video. There is also one for audio files. You may also be interested in system objects which provide more support here as well.
[Q]Are you planning to offer a webinar to read SEGY files?
[A]Currently no plans to offer one specifically on SEGY files. Our consulting services group often works with customers to solve SEGY file import questions (since there are many nonstandard formats out there)
[Q]do you have any documentation on this topic?
[A]You can find the format specification for SEGY standards on the web. Reading/writting from them is done using fread in MATLAB. There is extensive documentation on file i/o operaionts in MATLAB for text and binary file i/o
[Q]Do you use GeForce or TESLA GPU?
[A] We're using the TESLA GPU. To use GPU capabilities, you need to have a gpu that supports double precision, compute capability of 1.3. NVIDA TESLA series 10 and 20 are examples that work. Our website list more detials on supported GPUs.
[Q]Is GPU processing supported for Matlab Embedded for Simulink and RTW for xPC?
[A]There currently is no support for embedded applications to use the GPU.
[Q](re:wavwrite) is this new support in 7.12? I'm running 7.11 R2010b and I don't see anything new in the Help as far as wavwrite/wavread is concerned. Could you specify what you mean by "new support" and point me in the right direction?
[A]Look for system objects, it's supported in signal processing toolbox which is why you may not see it if you don't have this toolbox. Also look for MultimediaFileWrite which supports it as well.
[Q]Are the codes going to be posted under recent files (in matlab central)?
[A]Yes, they will appear there.
[Q]When you talked about 56 cores, did you mean multiple cores on multiple machines, or it is all about one machine?
[A]It was 56 cores spread across multiple machines, the max of which had 8 cores on each. 56 was the available number, we have a 64 core capability but some machines weren't available at the time.
[Q]Is a special toolbox required to use the gpu functionality?
[A]Parallel Computing Toolbox provides GPU functionality.
[Q]what are symptoms that the GPU is overloaded with too many threads or too much memory
[A] It would generally show up a slower processing speeds, but depends on your particular problem you're trying to solve on the GPU. Not all problems scale well with GPUs.
No comments:
Post a Comment