Identifying Feature Importance Using Multi Instance Learning (MIL)

You could also try using random forests. There are 2 fairly standard ways of estimating feature imprtance with RFs: one is base on the Gini coefficient reduction at the nodes that split on that feature & the other is based the performace degradation caused by completely randomizing that feature. The second way should really be applicable to any ML algorithm, but I have only seen it used in connection with RFs. RFs are very sensitive to class imbalace though, even a small imbalance can push an RF to just guesssing the most common class. Usually though subsampling the more common class to match the less common one seems to get good result. Alternatively you can include all the data in your training set, but undersample the more common class when constructing bags to train the individual trees. For undersampling during bagging, I have gotten sligltly better performace by sampling the bags without replacement. I think this is because if you sample with replaceant and are undersampling a much bigger class those samples will tend to be distinct, while sample from the less common class will contain duplicates, and fewer distinct examples. I think this ends up still biasing the learning. The randomForest package for R supports all of the above tweaks.

Is there any signifinace to the way your instances are grouped into bags or is it only to counteract the class imbalance? Can you just pool all the data into one training set? You colud also run the the feature importance algorithm on each of the multiple bags against the common bag & just see if the same features are important.