Here are some
half barely-baked project ideas based on recent papers I’ve read or things I’ve worked on recently:
- Extending Meshi et al. AISTATS 15, which uses soft constraints to make inference easier during training of structured SVMs, perhaps by using the same soft constraint idea for maximum likelihood.
- Paired-dual learning for learning latent variable discrete MRF models (instead of HL-MRFs, extending Bach et al. ICML15). E.g., how does it compare to the convex dual of Schwing et al. ICML ’12?
- Understanding the MAX-SAT linear programming relaxation (extending Bach et al. AISTATS 15 and related work), analyzing the guarantees in log space and how they jive with the hardness guarantees of MAP inference
- Back-propagation to compute gradients of PGM parameters for approximate inference (for max-margin or max likelihood learning), similarly to Zheng et al., arXiv 15
- Analyzing non-convex learning objectives in graphical models (latent variable learning, learning with Bethe-like entropies) in the context of Dauphin et al. NIPS 2014’s analysis (and related work) of saddle points. Do these objectives also suffer from a saddle point problem?
- Extending London et al. ICML 15 (and related work) to optimize counting numbers for entropy approximations for the learning task itself, not just as a regularization
- Can the method of moments / spectral learning enable very fast learning for templated graphical models? (E.g., see the bibliography from the 2014 workshop.)
- Regularizing structured prediction models by explicitly penalizing long-range dependencies (see Samdani & Roth ICML ’12, Torkamani & Lowd, ICML ’13, London et al. JMLR ’16)
- Exploring the relationship between logically-templated graphical models (e.g., MLNs) and causality. Are if-then potentials clues for causal relationships?
- AI search strategies for learning via searching (e.g., Daumé III et al. MLJ ’09), or applying AI search strategies to structure learning via searching through structures
- Extending “grafting” (see, e.g., Zhu et al, KDD ’10) for structure learning in logically-templated graphical models (where there is a clearly defined space of feature functions given the observed variables)
These ideas are not guaranteed to work or make sense.