I don't use ActiveAdmin much, but I do have RailsAdmin set up on Rails 4, no problems:

    $ grep \'rails Gemfile
    gem 'rails', '~> 4.0'
    gem 'rails_admin',  '~> 0.5.0'

https://github.com/sferik/rails_admin