About PoPETs Reviews

Review Process

Note: This information applies to PoPETs 2025.

The review process is a multi-stage process that begins after the submission deadline and is summarized in this flowchart:

The review process includes the following steps:

1. Desk Review: Submissions are quickly reviewed for length violations, author anonymity violations, and submission on topics that are out of scope. Submissions that violate these or other submission requirements will be “desk rejected” and will not receive a full review by the editorial board.
2. Full Review: A full review by the editorial board is completed in two rounds. Each submission is assigned an associate editor-in-chief to manage the review process. Editors are instructed to write detailed and constructive reviews of their assigned submissions and provide an initial decision recommendation.
   • Round 1: Submissions are assigned to at least two editors. Following the Round 1 review period, submissions that have received at least two reviews and for which all reviews recommend a reject decision may be rejected without further review and without the opportunity for rebuttal. Authors will be notified of this “early reject” decision by the beginning of the rebuttal period.
   • Round 2: PoPETs aims to provide four reviews for submissions that are not rejected early. To achieve this goal, generally two additional editors are assigned during Round 2 to review submissions that have not yet been rejected.
3. Rebuttal: Authors are sent the reviews written by the editors and then may submit a short 500-word rebuttal to answer the editors’ questions and concerns. The rebuttal period is around 3 days. The authors are given the chance to provide additional responses to reviews that come in late.
4. Discussion: The co-editors-in-chief, associate editors-in-chief, and editors participate in online discussions to come to a decision for each submission. The associate editors-in-chief are instructed to solicit opinions, drive consensus on the decision, and help the editors write meta-reviews summarizing the discussion and reasons for each decision. Editors may update their reviews based on the discussion.
5. Decisions: Decisions are finalized by the co-editors-in-chief, and the final reviews and meta-reviews are sent to authors. The next step for authors depends on the decision:
   • Accept: Authors should prepare a camera-ready version of their submission, taking into consideration the meta-review and the reviewers’ comments. No further review of the camera-ready version is done.
   • Revise: Authors are invited to revise their submission with the guidance of a revision editor according to a well-defined set of revision criteria included in the meta-review. The interactive revision process is outlined below.
   • Reject: Authors may choose to revise their submission according to the reviews and meta-review and resubmit a revision to a future issue of PoPETs. A summary of changes is required upon resubmission and must demonstrate that the authors have made substantial modifications to warrant a resubmission.
6. Interactive Revision: An interactive revision process will be used to enable authors to complete the revisions that the reviewers agreed must be completed and verified before the article can be accepted at PoPETs. A revision editor will be assigned to interactively guide the revision according to the following process:
   • Schedule: The authors will be required to propose a revision schedule within a few days of author notification, and the revision editor will approve or suggest modifications to the schedule as needed. The authors should consider the revision criteria from the meta-review and the following two deadlines when planning a schedule:
     • Early deadline: The current issue’s revision deadline is 1 month after author notification. Authors that complete the required revisions and have their revised submissions accepted before the early deadline will be published in the issue to which their article was submitted.
     • Final deadline: The next issue’s revision deadline is 4 months after author notification. Authors that complete the required revisions and have their revised submissions accepted before the final deadline will be published in the next issue following that to which their article was submitted.
   • Revision: Authors will revise their submissions according to the meta-review and their approved revision schedule. During this process, they may ask the revision editor for clarifications about the revision criteria listed in the meta-review, intermediate feedback, or further direction to prevent unnecessary work and ensure the planned revisions are appropriate. Authors will submit revised articles (including a latex diff) that they believe address the revision criteria to the revision editor, who will review the revision and provide additional feedback if necessary or make a final decision. Authors who attempted to meet the early deadline but failed may still meet the final deadline.
   • Final Decision: The revision editor may recommend a final decision of accept at any time before the early or final deadlines, and must recommend a final decision of accept or reject before the final deadline. The decision is then finalized by the co-editors-in-chief. Authors receiving accept will prepare a camera-ready version of their submission by the first camera-ready deadline subsequent to the final decision.

Review Decisions

Note: This information applies to PoPETs 2025.

Decisions are made on the basis of reviews, author rebuttals, and online discussion. Each submission is judged independently, and in particular PoPETs does not consider the total number of accepted papers when making decisions (i.e., there are no "accept if room" decisions). Submissions receive one out of three possible decisions.

Accept

An accept decision is given to submissions that can either be accepted as they are, or that require minor revisions that the editors trust can be made by the authors without being verified in a more lengthy revision process. Changes that may be in scope for accept include: editorial improvements (fixing grammar, punctuation, typos, or adjusting tables and figures), adding disclaimers on issues left out of scope, adding missing citations, etc.

Revise

A revise decision is given to submissions that we would like to eventually publish at PoPETs, but that require additional revision and review by an editor prior to publication. These submissions will be assigned a revision editor and follow the interactive revision process described above.

The required revisions may include changes that must be verified by an editor, such as the restructuring of some sections in the text, improving the motivation, toning down claims, including more details on methodology or experiments, providing missing clarifications, etc. The required revisions may also include more substantial changes that are needed to address important flaws in the submission prior to publication, such as performing additional experiments or analysis, demonstrating robustness against certain attacks, improving the security analysis, etc.

During the interactive review process, the revision editor will judge revised submissions according to the revision criteria specified in the meta-review. It is thus crucial that the meta-review provides a concrete list of revisions that would make the article acceptable if completed. The list should be as explicit and prescriptive as possible so that we can minimize the number of cases in which revised submissions that seem to address the revision criteria are still considered inadequate for PoPETs. The requested changes could be completed within one month, but should always be feasible within at most four months.

Acceptance of submissions receiving a revise decision is not guaranteed. A revised submission that does not adequately address the revision criteria specified in the meta-review, as judged by the revision editor, by the final revision deadline will be rejected.

Reject

A reject decision is given to submissions that the editors deem currently inadequate for PoPETs on the basis of the submission suffering from issues of novelty or lack of contribution, with fatal technical flaws that are non-trivial to address, or being out of scope for the topics of the venue (e.g., general security or theoretical cryptography articles that fail to clearly tie their contributions to improving or understanding privacy in real systems).

A reject decision typically signifies that substantial revisions are necessary for the article to be acceptable at PoPETs. The needed revisions are generally too large or their outcome too unpredictable that the editors were unable to construct a concrete, feasible list of changes that they believed would likely lead to an acceptable revision.

While the structure of PoPETs does not forbid submitting any article to a future issue, it is strongly recommended that authors receiving a reject decision take the necessary time to substantially improve their article. Thus, it is expected that any resubmission of a rejected submission will likely be delayed by more than one issue.

Who reviews papers for PoPETs?

Note: This information applies to PoPETs 2025.

PoPETs has a Program Committee (PC) / Editorial Board (EB) composed of internationally-recognised experts, who are invited to the PC/EB by the current PC Chairs. The PoPETs editors provide most of the reviews, with each article being reviewed by at least 2 editors. The review load is limited, typically to 2–4 papers per editor per issue. Editors have access to all non-conflicted submissions, and they are welcome to review or discuss papers that have not been assigned to them. Editors are expected to provide timely, high-quality reviews, actively participate in the discussion, contribute to reaching a consensus decision, and will be expected to serve as a revision editor for one or two submissions in each issue.

We ask editors to not delegate their reviews.

PoPETs may on rare occasions use external reviewers to provide needed expertise. If, as an editor, you are aware of an expert reviewer who would be perfect for a particular paper, please let the Chairs know and we can add them as an additional external reviewer (after checking for conflicts). External reviewers only have access to articles assigned to them for review, and are bound to confidentiality with respect to the article, the reviews and the discussion, in the same way as editors.

PoPETs provides an opportunity for junior researchers in particular to contribute as external reviewers. By "junior researchers" we mean recent graduates (postdocs) and senior PhD students who have at least 2 years of research experience, one or more publications in their topic, and the ability to review papers in their area of expertise. While junior researchers are often involved in peer review at other venues through sub-reviewing or "shadow PCs", we believe that direct participation in the real reviews and discussion with senior editors is the most effective way to train and engage newer researchers.

Please nominate excellent junior researchers to be external PoPETs reviewers through the online form. It is important to be specific in the keywords of the research areas as these are used to find matches between reviewers and papers. Self-nominations are welcome. Accepted nominees will be normal external reviewers, expected to provide reviews, participate in the discussion, and contribute to reaching a decision.

Review Guidelines

Note: This information applies to PoPETs 2025.

Timeliness:
The review process for an issue consists of: bidding, individual reviews (typically four per paper), author rebuttals, discussion, consensus decision, writing of meta-reviews, and possibly an interactive revision. Given that reviews are released to authors for the rebuttal phase, it is critical that they are not delayed. We also ask reviewers to please participate actively with comments during the discussion phase.

Thorough and substantive reviews:
We expect reviewers to write thorough reviews and not just a couple of short paragraphs. By limiting the review load, we hope that editors can focus on writing more substantive reviews based on a deeper reading of the paper.

Human-written reviews:
We expect that reviewers will not use AI-based tools (e.g. ChatGPT, Copilot) to write or edit their reviews, even for fixing grammatical mistakes. Furthermore, the reviewers are prohibited from inputting or uploading the submitted articles, or draft reviews that may contain details about the submitted articles, to AI-based tools. At the time of review submission, reviewers are asked to self-certify that the review they are submitting is written by themselves and not generated or edited by AI-based tools.

Constructiveness:
While many papers will no doubt be rejected, authors should know that their papers were turned down for reasonable issues. The authors may disagree with the final decision, but they should feel like the reviewers provided reasonable and constructive feedback. We thus ask reviewers to focus on the positives of the article, and try to be as constructive as possible so that authors can improve their work, regardless of whether they eventually receive an accept or reject decision.

Topics:
We are (continually) trying to broaden the scope of PETS and we are encouraging submissions from diverse areas. We have a diverse EB so that we have relevant reviewers for these areas. Still, the paper should relate to privacy (see the topics listed in the CFP for examples). We've included a review field related to relevance to PETS. Reviewers should comment on the relevance score in the review, especially if they gave it a low relevance score.

Novelty claims:
While it is reasonable to discuss how exciting and interesting are (or not) the results of a submission, if a reviewer states that the novelty of a paper is limited because the idea is not new (or has been known), we ask them to provide citations to back up this claim to show that the work has been published before.

Authors' Rebuttal:
The authors will have a chance to rebut/answer reviewer concerns/questions through a short rebuttal phase. If a reviewer's recommendation hinges on answers to certain questions, these should be specified in the review so that the authors may answer them in their rebuttal. Reviewers are asked to take the rebuttals into consideration during the discussion, and revise their review accordingly.

Discussion:
Reviewers are expected to participate in the online discussions for their papers. They should consider the opinions of other reviewers with the goal of coming to a consensus on a decision for a paper. Reviews should be updated to include any substantial changes in the reviewers' understanding or opinions so that authors can understand their reviews to reflect the views of the reviewers after rebuttal and discussion. The co-editors-in-chief are responsible for finalizing the decisions.

Vice Chair / Associate Editor-in-Chief:
Each submission will be assigned a Vice-Chair / Associate Editor-in-Chief (AE) to manage a submission's reviews and steer the discussion. The AE is also responsible for drafting a meta-review that summarizes the main suggestions for revision brought up in the reviews and discussion. This meta review should provide clear constructive suggestions for the authors on how to improve the paper depending on the decision: for revisions, clarify what revisions are expected; for rejected papers, clarify the main reasons why the paper is being rejected and what the authors might want to improve for a better shot for a resubmission at PoPETs or elsewhere. The AE should also encourage authors to add a rebuttal response, to consider the rebuttal during the discussion, and to update their reviews to reflect significant changes in their opinions.

Resubmissions:
Articles that have been previously submitted to PoPETs must be accompanied by a summary of changes that demonstrates that substantial progress has been made since the previous rejection.

Publicly available pre-prints:
It is recognized that, at times, information regarding the identities of authors may become public outside the submission process (e.g., if a pre-print is published as a technical report); reviewers are asked to ignore this external information. If the paper was not published in formal proceedings, please do not look for additional contributions beyond those informally published reports. In these cases we have allowed authors to say they have such a report with an anonymized reference (which is really a NOP reference). Additionally in the discussion a reviewer can note, e.g., "accept only if same authors" if they are concerned about plagiarism issues.

Prior published work:
An article submitted to PoPETs must present original work not described in any prior publication that is more than 4 double-column ACM conference-style pages in length. A prior publication is a paper that has been accepted for presentation at a refereed conference or workshop with proceedings, or an article that has been accepted for publication in a refereed journal. If a PoPETs submission has overlap with a prior publication, the submission must cite the prior publication (in third person regardless of whether the work is done by the same authors), along with all other relevant published work, even if this prior publication is at or below the 4-page length threshold.

Ethical concerns:
Reviewers are asked to bring up any ethical concerns they may have about the study methodology, regardless of whether the study received approval from an ethics board. Examples of research for which ethical issues may arise include articles describing experiments with users or user data (e.g., network traffic, passwords, social network information) and articles reporting privacy or security vulnerabilities in real-world systems. PoPETs reviewers should consider the basic principles of ethical research, e.g., beneficence (maximizing the benefits to an individual or to society while minimizing harm to the individual), minimal risk (appropriateness of the risk versus benefit ratio), informed consent, respect for privacy, and limited deception. Authors are encouraged to include a subsection on Ethical Principles if human subjects research is conducted, and such a discussion may be required if deemed necessary during the review process.

Claims of Benefits to Particular Populations:
Like human subjects research and supporting statements from IRBs and similar, failure to support claims about usefulness for a target demographic should not be grounds for immediate rejection. If this isn't in the article, reviewers should ask for it to be added/clarified before assuming the appropriateness of the proposed system/technology for that group. Authors should be expected to address this in their rebuttal. And if reviewers don't feel competent to judge whether the claims about a system's benefit to a particular user population are appropriate, it might be a place to suggest seeking an external reviewer (if the article is otherwise competitive enough).

However, progress and understanding sometimes does result from simplifying assumptions. Basic research work that makes clear that it is merely speculating usefulness for a particular user population and indicates potential problems and questions, and the need for validation of applicability, should still be accepted if the theoretical contribution is solid.

Systematization of Knowledge (SoK) articles:
We accept Systematization of Knowledge articles. Such articles critically review, evaluate, and contextualize work in areas for which a body of prior literature exists, and whose contribution lies in systematizing the existing knowledge in that area. To be suitable for publication, SoK articles must provide an added value beyond a literature review, such as novel insights, identification of research gaps, or challenges to commonly held assumptions.

Security Proofs and Lengthy Appendices:
In most cases, reviewers are not required to read or review content in the appendix. Some articles require lengthy security proofs to support the technical validity of the contribution. These articles should indicate this in the body of the article and include the proof in the appendix. Reviewers are not required to read the appendices at this phase (though they are welcome to do so if they wish). If the article would receive an accept decision pending a valid proof, one or two experts would be asked to review the proof. If the proof is lengthy this may result in delayed notification and publication for such articles (as the authors have been warned).