The intention of this group is to bring together members of the ASKAP Science Data Processing (SDP) group and members of ASKAP science survey teams including EMU, POSSUM, VAST and anyone else who is interested in joining.
Our current goals are to contribute to commissioning the end-to-end ASKAPsoft pipeline and to ensure that it is meeting the needs of our early science data requirements. I encourage all survey teams to participate in this process since we are all reliant on the performance of the up-front stages of the pipeline. Use of ASKAPsoft is not mandatory, since anything we can learn about the ASKAP data and how to most effectively deal with it will be useful, but I encourage everyone to try out the end-to-end processing pipeline or at least to familiarize themselves with its capabilities.
Some useful resources are linked below for those who have access:
A list of group members is maintained here.
Please help to keep this page up to date and provide a little description of what you are actively working on.
A number of active projects and ideas for future projects is maintained here.
I will compile a table below with candidate test data sets that I suggest we focus on.
I will link each data set to its own page so we can discuss our results and compare notes.
| Field Name | SBID | Cal SBID | Notes |
| LMC | 1997 | 1995 | closepack36 footprint. only 30 beams ingested. A-B-C interleaving. 10+ hours. 48 MHz BW @ 1400 MHz |
| CDFS | 2102 | 2099 | square_6x6 footprint. 36 beams. A-B interleaving. 10+ hours. 48 MHz BW @ 1032 MHz |
| CDFS | 2107 | 2104 | square_6x6 footprint. 36 beams. A-B interleaving. 10+ hours. 48 MHz BW @ 1032 MHz |
| GAMA23 | table | Several data sets from Dec 3-18, not without their issues. see table for full descriptions |
I created a directory on the Pawsey file system where we can share our scripts, imaging results, intermediate data products, etc.
/group/astronomy856/askapdp
Please create sub-folders before contributing your results to help keep it organized.
I will maintain a brief summary of each meeting below:
28Sept2016
At our kickoff meeting we talked about what our goals were for this group, which seemed very well aligned in terms of hands-on work and collaborative efforts. Meeting weekly for about 45 minutes was widely supported. It was suggested that we include some time in each meeting for hands-on work.
We decided that for the next couple of weeks we would combine our efforts on the earliest stages of the pipeline, i.e., flagging and solving for the bandpass of the 1934 calibration observations. We chose to take a close look at SBID 1995 (see link for more details). Wasim and Josh will also work within ACES to propose a long (~8 hour) calibrator observation to better understand bandpass stability.
04Oct2016
We discussed moving the meeting to Tuesdays at 2:30 AEDT. An email was sent around asking for feedback about this.
We talked about the antenna-time flags written by ingest and their potential effects on sensitivity.
We looked at how including the short baselines affects the bandpass solution.
We looked at how successful cflag was in dealing with RFI and other data quality issues. Some concerns about low-level RFI and baseline slopes.
We talked about smoothing and flagging the bandpass table and the possible origin of some spurious solutions.
We thought about possible improvements during beam forming that could make the solutions smoother across beam forming intervals.
We decided to continue focusing on the flagging and bandpass stages of the pipeline for the next couple weeks.
15Nov2016
We mostly reviewed the flagging situation and decided to move on to the steps needed to image the target field.
-whether or not to incorporate the Stokes-V flagger into the pipeline.
-how to distribute time for interleaved fields
-will a single set of pipeline parameters work for most (80% ?) of fields?
-clean's small scale bias parameter
-different weighting scheme for galactic plane / LMC?
-what limits the sensitivity of deep continuum images? RFI, shallow deconvolution, calibration errors, etc?
-details of selfcal tuning, especially model construction
06Dec2016
-new pipeline features, especially the variation of parameters per selfcal loop.
-should we request that selfcal respect scan boundaries
-how significant are beam-to-beam correlations for linmos weights
-need more investigation of solvers, especially basisfunction MFS
-validation of linmos of higher order terms, primary beam correction for tt1. validate against cube
-lots of primary beam efforts needed. fit to holography data, implementation in linmos